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ABSTRACT 


In  visual  simulations,  the  human  must  make  most  of  her  decisions  based 
on  the  visual  cues  rendered  to  her  display.  On  the  other  hand,  synthetic  forces 
have  the  luxury  of  basing  their  decisions  on  the  data  contained  in  the  simulation’s 
model.  Line  of  sight  calculations  are  often  examples  of  the  synthetic  player’s 
excess  of  information.  Current  methodologies  for  determining  a  synthetic 
player’s  line  of  sight  to  a  target  are  generally  variations  of  a  ray-casting 
technique.  Hiding  from  a  synthetic  player  in  plain  sight  by  using  shadow, 
camouflage,  or  by  simply  remaining  motionless  is  not  possible.  Synthetic  vision 
is  an  alternative  to  ray-casting.  We  perform  multiple  renders  from  each  synthetic 
player’s  point  of  view  and  temporarily  maintain  those  images  in  graphics 
memory.  We  then  execute  vertex  and  fragment  shader  programs  to  make 
comparisons  of  the  stored  images.  All  the  renders  and  calculations  are 
performed  on  the  Graphics  Processing  Unit  (GPU)  and  the  result  is  returned  to 
the  synthetic  player  in  the  form  of  an  annotated  list  of  visible  targets.  Performing 
these  target  visibility  calculations  on  the  GPU  gives  the  synthetic  player  a  more 
robust  spectrum  of  visual  inputs  with  which  to  make  decisions,  enabling  more 
realistic  behaviors. 
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I.  INTRODUCTION 


A.  STATEMENT  OF  THE  PROBLEM 

1.  Disparity  of  Information  Access 

In  three-dimensional  visual  simulations,  the  human  players’  visual  input  is 
generally  limited  to  an  image  rendered  to  a  display.  The  display  can  be  any 
device  from  a  standard  CRT  monitor  to  a  head-mounted  device  or  a  multi-screen 
“cave.”  In  any  case,  the  image  displayed  is  almost  always  a  two-dimensional 
representation  of  the  player’s  field  of  view  in  the  simulated  world.  The  human 
must  make  most  of  her  decisions  based  on  the  visual  cues  garnered  from  this 
image  on  her  display. 

Not  only  is  the  image  two-dimensional,  removing  the  three-dimensional 
cues  humans  normally  rely  on,  but  the  field  of  view  is  also  greatly  reduced 
compared  to  normal  vision.  For  example,  a  player  whose  eyes  are  21  inches 
(51 .45  cm)  from  a  17-inch  monitor  is  only  realizing  a  vertical  field  of  view  of  54.6° 
and  71.6°,  horizontally,  into  the  simulated  world.  Compared  to  a  human’s  normal 
vertical  and  horizontal  fields  of  view  of  approximately  90°  and  190°,  respectively, 
we  see  that  the  human  player’s  visual  inputs  from  the  simulation  are  a  fraction  of 
those  the  player  normally  enjoys.  This  deficiency  is  often  reduced  by  adding 
information  to  the  player’s  display  in  the  form  of  a  heads-up  display. 

On  the  other  hand,  computer-generated,  or  ‘synthetic’,  entities,  whether 
friend  or  foe,  have  the  luxury  of  basing  their  decisions  on  the  entire  body  of  data 
contained  in  the  simulation’s  model.  Without  appropriate  limitations,  the 
synthetic  force  can  potentially  be  omniscient.  For  example,  a  synthetic  player 
with  unlimited  access  to  information  can  know  the  location  and  status  of  any 
other  player,  human  or  synthetic.  Any  restrictions  on  a  synthetic  player  in 
accessing  information  from  the  model  are  aspects  of  the  synthetic  forces 
intentionally  engineered  by  the  simulation’s  designers.  This  filtering  or  hiding 
information  from  the  synthetic  force  is  a  challenging,  but  necessary,  requirement. 
The  synthetic  player  has,  by  default,  access  to  100  percent  of  the  model’s 
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information.  Without  limitations,  the  synthetic  player  will  have  an  excess  of 
information  and  will,  most  likely,  behave  unconvincingly. 

2.  Line  of  Sight  Calculations 

Line  of  sight  calculations  can  often  be  examples  of  a  synthetic  player’s 
excess  of  information.  Current  methodologies  for  determining  a  synthetic 
player’s  line  of  sight  to  a  target  are  generally  variations  of  a  ray-casting 
technique.  Ray-casting  involves  defining  a  line  segment  between  a 
predetermined  point  relative  to  the  three-dimensional  model  of  the  synthetic 
player  (its  “eye”)  in  the  simulated  world  and  a  predetermined  point  on  another 
model  -  the  target.  If  the  ray  between  those  two  points  does  not  intersect  any 
other  polygon  in  the  model,  whether  terrain  or  any  other  object,  then  the  ray 
passes  and  the  synthetic  player  is  considered  to  have  line  of  sight  to  the  target 
[Proc04].  Using  a  one-dimensional  point  to  determine  the  visibility  of  a  three- 
dimensional  target  is  a  fairly  simple,  if  not  over-simplified,  model.  An 
underdeveloped  implementation  of  ray-casting  ineffectively  filters  the  abundance 
of  information  available  to  the  synthetic  player. 

Several  methods  are  used  to  make  this  method  more  reasonable.  One 
method,  for  example,  is  to  cast  multiple  rays  per  target.  The  target’s  silhouette  is 
simplified  to  a  two-dimensional  polygon  and  rays  are  cast  from  the  synthetic 
player  to  each  vertex  of  the  polygon  and  to  the  polygon’s  center.  The  weighted 
number  of  passing  rays  are  compared  to  some  threshold.  If  fewer  rays  than  the 
threshold  pass,  the  target  is  considered  to  not  be  visible.  Regardless  of  the 
method,  ray  casting  depends  on  determining  intersections  of  lines  with  polygons. 
These  are  well  known  calculations,  but  not  trivial.  Additionally,  as  simulations 
use  increasingly  complex  rendering  techniques,  these  ray-casting  calculations 
also  become  increasingly  complex. 

Consider  the  situation  where  a  synthetic  player  is  on  one  side  of  a  pane  of 
glass  facing  a  target  on  the  other  side  of  the  pane.  A  ray  cast  between  the  two 
players  will  intersect  the  polygon  that  is  used  to  model  the  pane  of  glass, 
indicating  the  target  is  not  visible.  In  order  to  mediate  this  disparity,  additional 

calculations  must  be  made  to  determine  if,  at  the  point  of  intersection  with  the 
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ray,  the  polygon  is  opaque.  (A  pane  of  glass  might  be  assumed  to  be  uniformly 
transparent,  but  perhaps  it  is  modeled  with  a  texture  of  dusty  corners,  or  a  pixel 
shader  is  used  to  create  glare  or  reflections.)  Now,  if  it  is  opaque,  the  ray  is 
indeed  intersected;  however,  if  it  is  not,  still  further  calculations  must  be 
performed  to  determine  to  what  degree  the  polygon  is  transparent  at  that  point 
and  how  that  effects  the  overall  visibility  calculation.  Using  ray-tracing  becomes 
more  difficult  as  scene  complexity  increases. 

The  above  scenario  is  an  example  of  a  larger,  overarching  disparity-  ray¬ 
casting  is  a  mathematical  solution  to  a  perceptual  problem.  Target  visibility  is  not 
a  binary,  black-and-white  problem.  It  is  a  problem  fraught  with  many  shades  of 
grey.  Ray-casting  uses  line  of  sight  to  determine  visibility,  but  takes  no 
environmental  effects  into  account.  There  is,  generally,  no  consideration  given  to 
effects  that  make  a  target  more  or  less  visible  as  compared  to  her  background 
such  as  contrast  or  camouflage.  Hiding  from  a  synthetic  player  “in  plain  sight”  by 
using  shadow,  camouflage,  or  by  simply  remaining  motionless  is  not  possible. 
Again,  the  synthetic  player’s  information  is  not  sufficiently  being  filtered,  and 
unnaturally  observant  behavior  emerges. 

Also,  as  a  mathematical  and  iterative  solution,  ray  casting  is  very  discrete. 
That  is,  at  its  core,  ray  casting  is  determining  the  visibility  of  an  object  by 
sampling  the  visibility  of  a  few  points  on  that  object.  Consider  a  ray  tracing 
algorithm  that  uses  a  computationally  expensive  12  rays  to  determine  visibility  of 
a  humanoid  target.  In  an  environment  of  moderate-to-dense  vegetation,  all  of 
those  12  discrete  points  on  the  target’s  model  can  easily  be  obscured,  especially 
if  the  target  is  a  moderate  distance  from  the  observer.  Consider  that  a  threshold, 
say  8  of  the  12  rays,  for  example,  is  normally  used  to  determine  visibility.  We 
can  see  that  a  target  is  even  more  likely  to  be  mistakenly  considered  invisible 
even  if  in  stark  contrast  to  its  surroundings.  In  general,  ray  casting  has  the  ability 
to  not  only  give  too  much  information  to  the  synthetic  player,  but  can  also  provide 
too  little.  The  solution  to  both  extremes  lies  in  using  additional,  available 
information  to  determine  a  target’s  visibility. 
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3.  Purpose  of  the  Study 

Synthetic  Vision  is  an  alternative  to  ray-casting  in  determining  visibility  of 
targets  in  a  three-dimensional  simulation.  By  rendering  a  scene  for  the  synthetic 
player  just  as  is  done  for  human  players,  we  more  closely  limit  the  synthetic 
player’s  amount  of  information  to  that  of  the  human  player.  Instead  of  having  100 
percent  of  the  available  information  and  having  to  pare  down  from  there,  our 
synthetic  player  begins  with  a  limited  amount  of  information  and  gathers  only  that 
additional  information  which  is  necessary.  This  process  is  somewhat  analogous 
to  the  human  player’s  limited  field  of  view  augmented  with  a  heads-up  display. 

Synthetic  Vision  is  the  implementation  of  an  architecture  with  which 
synthetic  players  determine  visibility  of  possible  targets.  It  involves  performing 
multiple  renders  from  each  synthetic  player’s  point  of  view  and  temporarily 
maintaining  those  images  in  graphics  memory.  Vertex  and  fragment  shader 
programs  are  executed  to  make  comparisons  of  the  stored  images.  All  the 
renders  and  calculations  are  performed  on  the  Graphics  Processing  Unit  (GPU) 
and  the  result  is  returned  to  the  synthetic  player  in  the  form  of  an  annotated  list  of 
visible  targets.  Performing  these  target  visibility  calculations  on  the  GPU 
accounts  for  and  is  mediated  by  visual  properties  (lighting,  texturing,  and 
shading)  rather  than  dumb  rays.  This  gives  the  synthetic  player  a  more  robust 
spectrum  of  visual  inputs  with  which  to  make  decisions,  enabling  more  realistic 
behaviors. 

4.  Research  Questions 

a.  Feasibility  of  Impiementing  the  Architecture  Without 
Using  the  GPU 

Can  the  algorithm  be  implemented  using  the  Central  Processing 
Unit  (CPU)  only?  Can  the  synthetic  player’s  viewpoint  be  rendered  to  graphics 
memory  and  can  comparisons  be  made  of  those  generated  images  in  order  to 
determine  the  visibility  of  targets  within  the  player’s  field  of  view? 
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b.  Feasibility  of  Implementing  the  Architecture  Using  the 
GPU 

Assuming  that  the  synthetic  player’s  viewpoint  can  efficiently  be 
rendered  to  graphics  memory;  can  comparisons  be  performed  on  the  GPU?  Is 
there  a  performance  gain  and,  if  so,  under  what  circumstances  is  this  gain 
maximized? 

c.  Possible  Algorithms  for  Making  Comparisons 

What  are  some  possible  algorithms  that  can  be  used  to  compare 
the  rendered  images?  Can  they  be  implemented  accurately  and  efficiently?  Will 
these  algorithms  be  original,  taken  from  computer  vision  theory,  or  hybrids? 

B.  LIMITATIONS  OF  THE  STUDY 

The  domain  of  this  thesis  is  three-dimensional  virtual  simulations.  One  or 
more  human  players  participate  in  the  simulation  through  an  input  suite  and 
some  type  of  display,  e.g.  a  personal  computer.  Synthetic  forces  participate  in 
the  simulation  as  the  human  players’  teammates  and/or  opposition.  The  ability  of 
a  synthetic  player  to  determine  the  visibility  of  other  players  in  its  field  of  view, 
whether  synthetic  or  human,  is  the  focus  of  this  thesis.  In  particular,  the 
investigation  of  an  architecture  upon  which  different  visibility  algorithms  can  be 
implemented  is  the  goal. 

There  are  many  factors  involved  in  the  manner  in  which  humans  detect 
and  identify  objects  within  their  field  of  view.  This  study  does  not  approach 
modeling  the  identification  process.  The  development  of  a  suite  of  algorithms  to 
be  used  in  detection  is  also  out  of  the  scope  of  this  thesis.  We  implemented  two 
algorithms  for  the  purpose  of  determining  feasibility  of  implementation  and  also  to 
gather  insights  into  the  efficiency  of  an  implemented  algorithm. 

C.  THESIS  ORGANIZATION 

The  remainder  of  this  thesis  will  be  organized  as  follows: 

•  Chapter  II.  Current  Methodologies.  We  will  look  at  and  evaluate 
some  current  methodologies  for  determining  detection  (generally 
assuming  line  of  sight).  Of  particular  interest  is  the  US  Army’s 
ACQUIRE  model  for  target  detection. 
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•  Chapter  III.  Technology  Review.  Chapter  III  will  describe  the 
central  technologies  used  in  implementing  Synthetic  Vision  to 
include  Pixel  Buffers,  Rendering  to  Texture,  and  Shading 
Programs. 

•  Chapter  IV.  Implementation.  The  Synthetic  Vision  algorithms  and 
implementation  will  be  described  in  detail  for  both  the  “proof  of 
concept”  application  and  a  proposed  library. 

•  Chapter  V.  Testing  and  Results.  Two  types  of  testing  are  required 
to  evaluate  the  research  questions  listed  above:  feasibility  and 
time.  These  are  evaluated  for  our  two  types  of  algorithms 
implemented  in  the  demonstration  application  described  in  Chapter 
IV. 

•  Chapter  VI.  Conclusions.  The  implementation  and  results  are 
briefly  revisited  and  we  discuss  some  of  the  successes  and 
shortcomings  of  Synthetic  Vision,  in  our  estimation.  A  few 
recommendations  for  further  investigation  round  out  this  chapter. 
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II.  CURRENT  METHODOLOGIES 


A.  INTRODUCTION 

Different  approaches  have  been  taken  to  achieve  the  same  goal  of  more 
realistic  target  detection  and  identification.  Realistic  is  here  defined  as  similarity 
to  human  target  detection  performance.  In  fact,  determining  whether  a  synthetic 
entity’s  detection  behavior  is  similar  to  a  human’s  is  difficult  in  itself.  There  is 
simply  very  little  data  representing  the  multitude  of  variables  involved  in  this 
human  perception  task. 

What  has  been  done  in  earnest  is  to  model  the  detection  process  for 
electronic  sensors.  As  described  by  [Johnson58],  the  performance  capability  of  a 
sensor  (in  his  specific  case,  image  intensifiers)  can  be  modeled,  given  a  number 
of  variables.  The  US  Army’s  Night  Vision  Electronic  Sensor  Division  (NVESD) 
used  this  algorithm  to  generate  models  for  a  number  of  sensors  versus  lighting 
conditions.  This  is  the  US  Army’s  ACQUIRE  model  for  target  detection  and 
identification  in  high-resolution  visual  simulations.  NVESD  went  further  to 
interpolate  their  model  to  include  the  unaided  human  eye  and  sunlight  [NVESD2]. 

Versions  of  ACQUIRE  appear  in  the  QneSAF,  JANUS,  and  Combat^^' 
simulations,  all  of  which  are  currently  being  used  or  developed.  The  ACQUIRE 
model  is  also  modified  and  used  by  Champion,  Fatale,  and  Krause  to  model  line 
of  sight  in  vegetated  areas  [Champ96],  Lind  and  Uriels  to  design  a  prototype  line 
of  sight  algorithm  for  JANUS  (A)  [Driels95],  and  also  Reece  and  Wirthlin  in 
modeling  synthetic  player  target  detection  and  identification  for  the  Team  Tactical 
Engagement  System  [Reece96]. 

B.  BACKGROUND 

1.  Johnson  Cycle  Criteria 

Central  to  ACQUIRE  is  an  algorithm  developed  by  John  Johnson  of  what 
were  the  US  Army  Engineer  Research  and  Development  Laboratories  in  Fort 
Belvoir,  Virginia.  Johnson’s  idea  was  that  when  using  electro-optics,  the  output 
of  the  system  was  a  visible  image  that  a  human  observer  can  use  for 
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interpretation  and  decision  making  [Johnson58].  Further,  he  discretized  the 
levels  of  discrimination  to  No  Detection,  Detection,  Shape  Orientation,  Shape 
Recognition,  and  Detail  Recognition.  Johnson’s  Cycle  Criteria  is  dependent 
upon  how  many  resolvable  cycles  can  be  determined  across  a  function  a  two- 
dimensional  projection  of  the  presented  area  of  a  target.  “This  concept  assumes 
that  a  target  is  characterized  by  a  critical  target  dimension,  which  contains  the 
target  detail  essential  to  discrimination.”  [NVESD1].  Electro-optic  sensors  have  a 
finite  resolution.  This  resolution  is  defined  by  a  frequency,  or  number  of  cycles, 
that  fit  within  the  field  of  view  of  the  sensor.  This  is  analogous  to  defining  the 
resolution  of  a  computer  monitor  by  the  number  of  rows  of  pixels  the  monitor  can 
display.  The  resolvable  cycles  mentioned  above  refers  to  the  number  of  these 
finite  units  that  span  the  critical  dimension  of  the  target  in  the  sensor’s  field  of 
view.  This  is,  basically,  using  a  function  of  the  resolution  of  the  optics  and  range- 
to-target  to  determine  the  possible  level  of  detail  the  human  observer  will  realize 
based  on  the  size  of  the  target’s  critical  dimension,  typically  the  minimum 
dimension,  in  the  observer’s  view.  Again,  this  is  a  model  designed  to  predict 
performance  in  electro-optical  sensors  where  resolvable  cycles  and  other 
variables  can  be  quantified  by  field  testing. 

This  testing  was,  in  fact,  done  by  NVESD  to  generate  criteria  for  the 
discrimination  of  targets  of  interest  for  the  various  levels  of  discrimination 
[NVESD1].  The  US  Army’s  standard  Contrast  Model  is  an  example  of  one  of 
these  criteria.  The  Contrast  Model,  is  a  mix  of  targets  and  backgrounds,  spectral 
data,  and  sensor,  filter,  and  source  characteristics  and  provides  values  for  the 
Inherent  Contrast  variable  to  Johnson’s  algorithm  [NVESD2].  While  the  number 
of  permutations  of  this  table  is  quite  large,  the  resulting  values  are  still  discrete. 
Any  variables  for  which  data  has  not  been  collected  must  be  extrapolated  from 
similar  known  values. 

2.  ACQUIRE  Model 

The  Army’s  current  standard  algorithm  for  modeling  Search  and  Target 
Acquisition  (STA)  is  the  ACQUIRE  model.  ACQUIRE  is  an  empirical  model 
based  on  Johnson’s  Cycle  Criteria  used  to  determine  target  acquisition 
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performance  for  imaging  systems.  It  is  able  to  predict  performance  for  three 
different  types  of  tasks:  target  spot  detection,  target  discrimination,  and  time- 
dependent  target  detection.  Target  spot  detection  is  based  on  a  signal-to-noise 
(contrast  or  temperature)  ratio  of  a  target  against  a  uniform  background.  Target 
discrimination  involves  Johnson’s  Cycle  Criteria  and  is  used  for  targets  against  a 
heterogeneous  or  cluttered  background.  Finally,  time-dependent  target  detection 
determines  the  probability  of  detecting  a  target  within  a  given  duration  of  time. 
The  model  was  originally  developed  by  the  NVESD  to  be  compatible  with  a  study 
of  the  long  range  use  of  forward-looking  infrared  sights.  Equations  appropriate 
for  direct  view  optics  (DVO),  such  as  binoculars  or  the  unaided  eye,  were 
extrapolated  from  the  original  model.  DVO,  however,  is  not  a  recommended  use 
of  ACQUIRE  [NVESD1]. 

Inputs  to  ACQUIRE  fall  into  four  categories:  Target  Characteristics, 
Environmental  Effects,  Sensor  Characteristics,  and  Task  Description  Inputs. 
Many  of  these  inputs  are  static  values  representing  unchanging  characteristics  of 
either  the  target  or  the  sensor.  Examples  of  these  are  light  level  and  the  Sensor 
Characteristics.  Many  other  inputs  such  as  apparent  signature  and  battlefield 
obscuration,  though,  are  dynamic.  These  values  are  generally  found  by  referring 
to  look-up  tables  for  known  values  or  interpolated  values  for  those  values  which 
are  unknown.  The  Standard  Contrast  Model,  described  above,  for  finding  the 
Target  Characteristic  of  Inherent  Contrast  is  an  example  of  one  look-up  table. 
How  these  dynamic  inputs  are  found  and  implemented  separates  the  various 
implementations  of  ACQUIRE. 

.  Ranges  and  probabilities  predicted  by  the  model  represent  the 
expected  performance  of  an  ensemble  of  trained  military  observers 
with  respect  to  an  average  target  having  a  specified  signature  and 
size.  [NVESD1] 

What  ACQUIRE  returns  as  outputs  are  a  list  of  the  portion  of  observers 
described  above  that  successfully  complete  each  acquisition  task  (spot  detection, 
discrimination,  and  time-dependent  detection)  as  a  function  of  the  observer-to- 
target  range.  For  each  task,  the  results  are  presented  in  two  formats:  probability 
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of  an  observer  to  be  in  the  portion  of  observers  who  successfully  complete  the 
task,  and  the  maximum  range  for  successfully  accomplishing  the  task  at 
probability  ranges  from  0.05  to  0.95. 

C.  RECENT  APPLICATIONS 

1.  Combined  Arms  and  Support  Task  Force  Evaluation  Model 
(CASTFOREM)  and  Janus  (A)  Training  Simulation 

CASTFOREM  and  Janus  are  both  entity-level  models,  that  is  they  model 
company  level  units  and  below  and  are  capable  of  modeling  single  entities,  such 
as  individual  vehicles  or  infantrymen.  They  both  use  the  ACQUIRE  model  to 
determine  probability  of  detection.  The  two  models  differ,  though,  in  their  use  of 
ACQUIRE;  Janus  uses  constants  as  ACQUIRE  parameters  which  remain 
unchanged  over  the  duration  of  the  entire  simulation  while  CASTFQREM  has 
more  dynamic  parameters.  Uriels  and  Lind  investigated  a  method  of  deriving 
dynamic  inputs  for  ACQUIRE  and  implementing  them  in  Janus.  An  algorithm  by 
Uriels  and  Lind  [Uriels95]  centers  around  ray  casting  over  a  regular  grid.  Their 
particular  focus  was  to  perform  their  derivations  in  a  database  driven  simulation 
using  a  perspective  view  generator  (PVG). 

The  PVG  creates  a  raster  of  pixels  representing  the  observer’s  point  of 
view.  Qne  ray  is  cast  per  pixel  from  the  point  of  view  at  a  calculated  offset  from 
boresight  until  it  encounters  the  ground,  a  target,  or  an  overhang.  An  overhang 
is  a  feature  under  or  behind  which  an  object  can  be  hidden.  This  ray  casting 
results  in  a  raster  of  pixels  on  the  framebuffer.  The  color  of  each  pixel  is 
determined  by  the  outcome  of  a  corresponding  ray  cast:  targets  are  red,  open 
terrain  is  a  grayscale  interpolation  of  the  relative  elevation,  and  areas  obscured 
by  an  overhanging  feature  are  colored  black.  This  framebuffer,  once  rendered  to 
the  display,  is  an  annotated  perspective  view  of  the  battlespace  from  the 
observer’s  point  of  view.  It  is  annotated  by  the  location  of  targets  in  red  and  all 
unobservable  areas  in  black. 

The  PVG  enables  some  of  the  inputs  to  CASTFQREM’s  implementation  of 
ACQUIRE  as  depicted  in  Figure  1,  such  as  Apparent  Contrast,  to  be  calculated 
dynamically,  increasing  the  realism  of  the  model.  The  ability  of  an  entity  to  enter 
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and  leave  unobservable  areas  by  were  a  great  improvement  over  previous 
implementations  of  ACQUIRE.  Driels  and  Lind’s  ability  to  incorporate  this 
algorithm  on  a  database  driven  simulation  allowed  them  the  luxury  of  calculation 
unaffected  by  the  density  of  objects  in  their  scene.  The  same  number  of  rays  is 
cast  for  a  complex  scene  as  a  simple  one;  it  is  dependent  only  on  the  dimensions 
of  the  observer’s  view. 


Figure  1.  CASTFOREM  Target  Detection  Algorithm  (From  Driel95) 


2.  Effects  of  Vegetation  on  Line  of  Sight 

Again,  predicting  line  of  sight  in  combat  simulations  realistically  is  a  key  to 
accurate  target  detection.  Before  Champion  et  al,  no  systematic  approach  was 
taken  to  quantify  the  effects  of  vegetation  on  line  of  sight  and  implement  them  in 
combat  simulations.  One  of  the  current  methods  of  modeling  these  effects  is  to 
partition  the  battlespace  into  categories  of  surface  features  and  have  different 
line  of  sight  criteria  for  each  feature.  For  example,  a  dense  forest  feature  may 
enable  line  of  sight  to  extend  only  3  meters  into  the  feature  while  the  rest  (its 
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interior)  has  no  line  of  sight  from  outside  the  feature.  Champion  points  out  the 
lack  of  quantitative  data  to  support  the  current  methods. 

US  Army  Training  and  Doctrine  Command  Analysis  Center  -  White  Sands 
Missile  Range  (TRAC-WSMR)  developed  a  study  to  identify  a  wide  variety  of 
vegetation  types  and  to  collect  data  within  each  area  in  order  to  determine 
percentage  of  target  visible  when  LOS  exists.  Their  goal  was  to  develop  a 
function  that  would  give  the  percentage  of  a  target  that  is  visible  to  an  observer 
given  the  surrounding  vegetation  type  and  LOS  range. 

This  percentage  coupled  with  the  range  of  the  LOS  is  used  to  determine 
the  resolvable  cycles  as  described  in  Johnson’s  Cycle  Criteria.  That  is,  the  ability 
for  the  current  optical  sensor  to  determine  detail  [Champ96].  It  is  somewhat 
intuitive  that  the  vegetation  surrounding  a  target  will  have  an  effect  on  the 
amount  of  it  an  observer  can  see.  The  less  intuitive,  aspect  of  this  model  is  that 
this  effect  is  implemented  by  modifying  the  characteristics  of  the  sensor  in 
Johnson’s  algorithm. 

For  demonstrative  purposes.  Champion  implemented  the  function  as  a 
modification  to  the  ACQUIRE  model  used  in  CASTFOREM.  Flis  example  is 
described  in  [Champ96,  pp  33-36].  As  usually  implemented.  Inherent  Contrast, 
Sky  Over  Ground  ratio,  and  Atmospheric  Attenuation  are  all  constants  from  a 
look-up  table  representative  of  the  location  of  the  scenario:  in  this  case,  Europe. 
His  results  show  a  significant  difference  between  the  probability  of  detecting 
soldiers  kneeling  and  on  the  move  and  those  prone  and  in  motion  at  ranges  from 
0  to  400  meters  under  various  light  conditions  [Champ96]. 

TRAC-WSMR’s  study  presents  an  enhancement  to  the  ACQUIRE  model 
by  introducing  vegetation-  and  climate-specific  modifiers  to  target  detection  and 
identification  processes.  Champion  identifies  the  requirement  for  the  inclusion  of 
factors  which  are  particular  to  a  geographic  region  in  simulation,  particularly  in 
the  realm  of  target  detection.  While  the  factors  for  determining  Resolvable 
Cycles  for  Johnson’s  Algorithm  are  now  more  easily  found  for  dismounted 
infantry  observed  with  the  human  eye,  other  possibly  dynamic  factors  are  still 
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described  as  constants.  In  Champion’s  example  application,  Apparent  Contrast 
is  calculated  with  three  of  its  four  inputs  defined  as  representative  constants. 

3.  Team  Tactical  Engagement  System  (TTES) 

TTES  is  an  application  that  was  being  developed  for  the  US  Marine  Corps. 
In  developing  visual  and  aural  detection  and  visual  identification  abilities  for  their 
“individual  combatants  (ICs)”,  Reece  and  Wirthlin  did  not  model  the  eye,  but 
modeled  characteristics  of  human  vision.  In  particular,  they  consider  the 
difference  between  the  highly  acute  1°  of  foveal  vision  and  the  less  acute  -95°  of 
peripheral  vision.  They  generalize  detection  in  the  foveal  and  near-foveal  (30°) 
areas  of  the  field  of  view,  simulating  visual  search.  In  general,  objects  in  the 
model  which  are  mathematically  determined  to  fall  inside  the  fan  described  by 
this  field  of  view  are  evaluated  for  visibility  and/or  identification. 


Figure  2.  Visual  Fields  of  View.  (From  Cham96  ) 

Detections  in  the  peripheral  field  of  view  are  effectively  immediate; 
in  this  30°  area,  however,  we  compute  an  acquisition  time  for 
detecting  objects  because  the  fovea  has  to  search  the  area.  This 
model  also  assumes  that  identification  is  immediate  once  the  fovea 
fixates  on  the  object.  [Cham96] 

The  IC  sighting  algorithm  used  is  based  on  Lind’s  sighting  model,  similar 
to  the  model  used  in  JANUS  and  ModSAF.  The  inputs  to  the  algorithm  include 
the  range  and  visibility  of  the  target,  environmental  variables,  attributes  of  the 
visual  sensor  (in  this  case,  the  modeled  field  of  view),  and  a  normally  distributed 
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random  number.  The  output  of  the  algorithm  is  a  sighting  status:  Invisible, 
Visible,  Detected,  Recognized,  or  Identified. 

Reece  acknowledges  limitations  in  their  algorithm.  Several  of  the  input 
variables  for  their  sighting  algorithm  are  constants.  In  particular,  “default  values” 
are  used  for  both  light  level  and  brightness  contrast  with  background.  Other 
limitations  of  the  study  are  outlined:  effects  of  low  light  levels  on  acuity,  effects  of 
observer  motion,  and  modeling  specific  direction  of  gaze  during  search.  These 
factors  are  understandably  difficult  to  define  and  quantify  solely  from  the 
simulation’s  mathematical  model.  However,  some  of  Reece’s  obstacles,  such  as 
“effect  of  contrast — brightness,  color,  and  texture”  and  specific  direction  of  gaze 
can  be  addressed  by  using  additional  information  that  can  be  gathered  from  the 
IC’s  rendered  field  of  view. 

D.  SUMMARY 

The  US  Army’s  ACQUIRE  model  and  Johnson’s  Cycle  Criteria,  indirectly, 
are  implemented  in  several  current  combat  simulations.  The  effort  to  make  this 
model  more  representative  of  target  detection  in  the  real  world  is  challenging,  yet 
ongoing.  The  above  examples  and  applications  of  ACQUIRE  have  two  aspects 
in  common:  they  still  require  global  or  at  least  very  discrete  values  for  some 
variables  and  they  are  all  in  the  domain  of  constructive  simulations. 

These  two  traits  are  surprisingly  related.  Constructive  simulations  are 
primarily  mathematically  based  models  with  associated  visualizations.  These 
simulations  manipulate  and  model  single  entities  on  the  battlefield,  but  these 
single  entities  are  simply  copies  of  one  discrete  mathematical  model. 
Accordingly,  each  interaction  between  similar  opponents,  with  the  imposed  slight 
randomness  aside,  is  nearly  identical  to  every  other.  For  example,  an 
engagement  described  as  one  soldier  engaging  another  soldier  will  be  calculated 
similarly  regardless  of  the  particular  instances  of  each  soldier.  The  variables  do 
not  lie  with  the  instances  of  soldiers,  but  in  the  environmental  factors.  These 
environmental  factors  are  also  very  discrete  with  constant  values  given  to  regions 
of  similar  vegetation.  A  soldier  in  an  area  labeled  as  ‘dense  vegetation’  engaging 

another  soldier  ‘in  the  open’  is  an  example  of  an  engagement  that  will  differ  from 
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the  one  described  above.  However,  if  this  engagement  occurs  a  second  time  at 
a  short  distance  from  the  first  engagement,  the  results  will,  again,  be  nearly 
identical  because  there  is  no  mechanism  to  discern  between  ‘this  dense 
vegetation’  and  ‘that  dense  vegetation.’  For  the  purposes  of  these  combat 
simulations,  this  phenomenon  is  completely  acceptable.  With  this  in  mind,  it  is 
understandable  that  the  variables  most  closely  coupled  with  the  individual  entity 
can  be  discrete  or  even  constant. 

This  thesis  experiments  in  the  domain  of  virtual  simulations,  though.  How 
can  the  evaluation  of  constructive  simulation  methods  of  target  detection  be  of 
any  interest  or  value?  Simply,  virtual  simulations  have  no  published  or  accepted 
standard  for  performing  target  detection  or  identification  outside  the  creative  use 
of  ray  casting  as  described  in  the  Introduction.  The  current  methodology  for 
target  detection  and  identification  in  both  games  and  first-person  simulations  is 
some  variation  on  ray  casting.  We  believe  that  we  can  use  information  available 
in  a  first-person  simulation  to  garner  a  more  complete  suite  of  inputs  analogous 
to  those  required  by  Johnson’s  algorithm.  Performing  calculations  involving  the 
color  and  depth  information  inherent  in  a  scene  rendered  from  the  observer’s 
point  of  view  can  result  in  accurate  and  dynamic  determination  of  target  area 
presented  and  the  apparent  contrast.  Of  particular  note,  the  literally  apparent 
contrast  takes  into  account  combat-induced  obscurants  and  atmospheric 
attenuation  if  these  facets  are  present  in  the  model. 
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III.  TECHNOLOGY  REVIEW 


A.  INTRODUCTION 

In  order  to  better  understand  our  implementations  of  Synthetic  Vision,  a 
basic  review  of  the  rendering  pipeline  is  provided.  Additionally,  we  describe 
some  of  the  other  technologies  used  in  developing  our  algorithm.  These  tools 
either  augment  or  circumvent  certain  portions  of  the  normal  rendering  pipeline. 
This  combination  of  methods  and  technologies  is  an  experimental  amalgamation 
and  while  the  individual  components  and  their  implementations  are  fairly  well 
known,  the  aggregation  of  these  components  is  not. 

One  example  is  our  use  of  the  GPU.  We  not  only  render  a  point  of  view 
off-screen  to  a  texture,  but  we  also  use  that  texture  as  an  input  to  another  off¬ 
screen  render  which  performs  visibility  calculations.  In  this  chapter,  we  will  take 
a  general  look  at  each  technology,  such  as  the  GPU.  In  the  following  chapter, 
we  will  describe  our  particular  implementations  of  each. 

B.  REVIEW  OF  THE  RENDERING  PIPELINE 

1.  Background 

Software  applications,  especially  real-time  simulations,  perform 
initializations  and  then  execute  some  form  of  a  run  cycle,  once  per  frame,  until 
the  application  is  terminated.  This  run  cycle  is  normally  composed  of  update  and 
draw  stages.  Inputs  to  the  system  are  generally  processed  by  an  event-handling 
scheme  and  can  be  considered  relatively  continuous  and  not  a  discrete  stage  of 
the  cycle.  In  describing  the  “rendering  pipeline,”  we  are  referring  to  the  draw 
stage  of  the  run  cycle.  This  is  the  process  of  taking  data  from  the  software 
application,  processing  that  information  into  a  geometric  representation,  and 
generating  a  visible  image  of  that  geometry  to  the  user  on  some  type  of  display. 
This  process  is  generally  implemented  in  today’s  commodity  hardware  by  at  least 
one  of  two  specifications:  the  Open  Graphics  Library  (OpenGL)  and  DirectX. 
OpenGL  is  licensed  by  Sililcon  Graphics,  Inc.  and  is  governed  by  the  OpenGL 
Architecture  Review  Board.  “End  users,  independent  software  vendors,  and 
others  writing  code  based  on  the  OpenGL  API  are  free  from  licensing 
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requirements.”  [OpenGL2]  DirectX  is  a  proprietary  standard  that  was  created  by 
Microsoft  for  rendering  in  Microsoft  Windows  applications.  The  DirectX 
specification  includes  many  aspects  of  game  development  to  include  input 
devices  and  audio.  Both  specifications  are  complied  with  in  nearly  all  the 
products  by  graphics  hardware  vendors,  nVidia,  ATI,  and  SDLabs,  for  example. 

Akenine-Moller  and  Haines  further  breaks  down  the  rendering  pipeline  into 
three  stages  in  [Aken02].  These  are  the  Application,  Geometry,  and  Rasterizer 
stages.  [ARB99]  and  [Rost04]  describe,  in  depth,  the  functions  of  the  latter  two 
stages.  This  brief  discussion  will,  similarly,  focus  on  the  last  two  stages. 
Geometry  and  Rasterizer,  from  the  perspective  of  OpenGL  with  emphasis  on  the 
latter. 

2.  Vertex  Processing 

The  goal  of  the  Geometry  and  Rasterizer  stages  is  to  take  the  geometric 
information  from  the  application  broken  down  into  a  stream  of  single  primitives 
and  process  them  for  display.  Primitives  are  defined  by  a  set  of  one  or  more 
vertices.  Each  vertex  has  a  color  and  location  in  object  space.  By  specification, 
all  primitives  are  closed  and  convex  and  are  recommended  to  have  coplanar 
vertices.  Triangles  are  most  commonly  used  since  any  group  of  three  points  are 
coplanar. 

These  geometry  primitives  are  passed  from  the  Application  stage  and  the 
positions  of  their  vertices  are  transformed  from  object  space  to  eye  space  to  clip 
space.  If  one  or  more  vertices  of  a  primitive  are  located  outside  the  intermediate 
clip  space,  they  are  discarded  and  vertices  are  created  at  the  boundary  of  the 
space  to  allow  partial  primitives  to  be  displayed.  The  final  transformation  is  to 
window  space,  described  in  detail  in  Chapter  3  of  [ARB99].  The  result  of  all 
these  transformations  is  a  location  in  window  space  of  each  ‘visible’  vertex  of  the 
primitive.  The  above  transformations  are  appropriately  referred  to  as  “vertex 
processing.”  These  locations  are  then  passed  along  with  each  vertex’s  color  and 
depth  information  to  the  Rasterizer  for  “fragment  processing.” 
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While  discussing  vertex  processing,  we  must  also  introduce  the  idea  of 
textures.  A  texture  is  an  image  which  is  applied  to  a  set  of  geometry.  A  simple 
example  is  a  pair  of  triangles  used  to  form  a  rectangle  over  which  a  single  texture 
is  placed.  Instead  of  the  rectangle  being  a  color  defined  by  its  vertices,  it  now 
has  an  image  applied  to  it,  like  a  decal  stretched  over  the  rectangle.  The  texture 
in  this  example  is  defined  to  span  both  triangles;  in  general,  a  texture  can  span 
either  a  single  primitive  or  span  several  contiguous  primitives.  Prior  to  the 
primitive  being  defined,  the  texture  image  is  stored  in  texture  memory  on  the 
video  card.  The  boundaries  of  the  texture  are  defined  by  a  set  of  texture 
coordinates.  Each  coordinate  is  coupled  to  a  corresponding  primitive  vertex. 
When  the  vertex  information  is  passed  down  the  pipeline  for  fragment 
processing,  the  corresponding  texture  coordinates  are  also  passed. 

3.  Fragment  Processing 

Since  all  primitives  are  closed  and  convex,  an  “inside”  region  can  be 
interpolated  from  the  vertices.  This  region  is  divided  into  discrete  units.  Each  of 
these  units  is  called  a  fragment  which  is  analogous  to,  but  not  always  equivalent 
to,  a  pixel.  A  pixel  is  a  color  sample  at  a  particular  point  on  an  image.  As 
described  quite  enthusiastically  in  [Smith95],  “A  pixel  is  not  a  small  square!” 
Fragments,  on  the  other  hand,  have  a  sense  of  area  in  that  they  are  later 
mapped  to  an  area  on  the  display.  For  primitives  without  textures,  the  color  and 
depth  of  each  fragment  is  interpolated  from  the  colors  and  depths  of  the  vertices 
composing  the  primitive.  If  a  primitive  is  textured,  the  color  value  for  each 
fragment  is  found  by  doing  a  look-up  on  the  texture  image  in  texture  memory. 
The  location  queried  is  based  on  an  extrapolation  of  the  primitive’s  texture 
coordinates.  Regardless  of  the  method  of  determining  the  color  value,  each  one 
of  these  fragment’s  color  and  depth  values  are  passed  to  the  Rasterizer. 

In  rasterizing  each  fragment,  several  tests  are  performed  to  determine  the 
color  and  visibility  of  each  fragment.  At  the  simplest,  if  the  depth  value  of  a 
particular  fragment  is  less  than  the  existing  depth  value  at  the  same  position 
(hence,  closer  to  the  eyepoint),  then  the  fragment  is  considered  to  be  visible.  Its 
depth  value  is  then  written  to  the  depth  buffer  at  the  appropriate  position(s)  and 
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its  color  value  is  similarly  written  to  the  frame  buffer.  This  process  is  done  for 
each  fragment  in  a  primitive  and  for  each  primitive  in  the  scene’s  geometry.  The 
result  is  an  array  of  color  values  in  the  frame  buffer  and  a  corresponding  array  of 
depth  values  in  the  depth  buffer.  Finally,  the  contents  of  the  frame  buffer  are 
mapped  to  the  display  device. 

This  discussion  was  a  brief  simplification  of  the  rendering  pipeline  for  the 
purpose  of  providing  some  background  to  the  following  sections.  There  are  a 
great  number  of  complexities  we  have  glossed  over.  These  include  double¬ 
buffering,  the  stencil  and  accumulations  buffers,  and  alpha  values.  Still,  we  now 
have  a  common,  basic  understanding  of  the  rendering  pipeline.  From  this  basic 
model,  we  will  now  diverge.  The  remaining  sections  will  discuss  the 
augmentations  and  alterations  to  this  model  which  are  required  to  be  understood 
before  moving  on  to  the  discussion  of  implementation  in  the  next  chapter. 

C.  PROGRAMMABLE  GRAPHICS  PROCESSING  UNIT  (GPU) 

1.  Background 

The  GPU  is  not,  in  itself,  a  new  technology.  It  is  the  microprocessor 
specifically  designed  to  process  three-dimensional  graphics  data.  On  August  31, 
1999,  nVidia  coined  the  term  with  the  unveiling  of  the  GeForce  256  video  card. 
While  ATI  later  coined  a  term  of  its  own.  Visual  Processing  Unit  (VPU),  GPU  is 
now  used  in  reference  to  all  modern  graphics  chips  [Prank].  The  processes 
described  above  are  referred  to,  loosely,  as  the  OpenGL  fixed  functionality 
pipeline.  This  behavior  is  prescribed  by  the  OpenGL  specification  version  1.1 
and  is  the  default  behavior  for  the  OpenGL  rendering  pipeline.  With  the  advent 
of  programmable  GPUs,  this  distinction  has  become  significant.  It  is  significant 
because  the  GPU’s  programmability  allows  users  to  alter  the  default  behavior  of 
the  pipeline  at  two  points,  vertex  processing  and  fragment  processing,  as 
depicted  in  Figure  3. 
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Figure  3.  Programmable  Pipeline  (From  Rost04) 


The  GPU  is  engineered  for  graphics  processing.  It  is  a  highly  parallel 
architecture  with  supporting  random-access  memory,  texture  memory,  frame 
buffer  memory,  and  a  cache.  All  of  this  supporting  architecture  resides  with  the 
GPU  on  the  graphics  card.  The  current  commodity  GPU  can  perform,  roughly, 
an  order  of  magnitude  more  operations  per  second  than  a  commodity  CPU.  In 
the  CPU’s  defense,  the  GPU’s  operations  are  quite  specialized  and  optimized. 
The  current  GPU  can  access  its  memory  much  quicker  than  CPU,  also.  Overall, 
the  processing  done  onboard  the  graphics  card  can  be  done  much  quicker  than 
on  the  CPU  for  many  operations. 

2.  Considerations 

A  very  important  aspect  of  the  GPU  to  consider  is  the  idea  of  mandatory 
parallelism.  As  discussed  previously,  the  input  to  the  GPU  is  a  stream  of 
primitives.  These  primitives  are  broken  down  into  vertices  and  then  further  into 
fragments.  This  is  where  the  parallelism  comes  into  play;  the  fragments  are 
processed  independently  and  in  parallel.  That  is,  every  fragment  processed  has 
no  information  regarding  the  fragments  that  came  before,  will  go  after,  or  any  of 
its  neighboring  fragments.  There  are  no  comparisons  between  fragments.  This 
enables  a  large  number  of  fragments  to  be  processed  in  the  amount  of  time 
required  by  the  single  slowest  fragment. 

One  particular  operation  for  which  the  GPU  is  optimized  is  random  look¬ 
ups  from  texture  memory.  While  it  can  not  write  directly  to  texture  memory, 
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accesses  from  that  memory  are  relatively  quick.  Texture  data  are  placed  in  the 
texture  memory  onboard  the  graphics  card  and  are  used  as  inputs.  If  few 
enough  textures  are  used  by  the  application,  these  textures  all  remain  ‘active’ 
and  are  immediately  accessible.  Otherwise,  textures  are  paged  into  and  out  of 
texture  memory  from  system  memory. 

3.  General  Purpose  Computing  on  the  GPU 

Using  the  GPU  to  perform  computations  other  than  rendering  is  certainly 
not  unique  to  this  study.  A  recent  workshop  for  discussing  this  very  enterprise, 
the  2004  ACM  Workshop  on  General-Purpose  Computing  on  Graphics 
Processors  (GP''2),  showcased  a  number  of  current  applications  and  studies 
varying  from  urban  gas  flow  and  dispersion  to  studying  molecular  dynamics  to 
the  work  described  in  this  thesis  [GP2].  There  are  even  application  libraries  upon 
which  applications  can  be  written  to  utilize  the  GPU  with  little  knowledge  of  the 
rendering  pipeline.  Stanford  University’s  BrookGPU  is  an  example  of  one  of 
these  libraries  [Brook]. 

Regardless  of  the  architecture,  the  problem  set  for  which  the  GPU  is  well- 
suited  is  constrained.  As  mentioned  above,  the  GPU  has  a  form  of  mandatory 
parallelism.  With  each  piece  of  data  being  processed  independently, 
applications  requiring  functions  as  commonplace  as  minimum,  maximum,  or 
even  sums  across  a  dataset  are  not  readily  programmed.  One  way  to  address 
these  problems  is  to  perform  additional  passes  of  the  pipeline  to  implement  any 
desired  comparison  operations  in  a  reduction  process  as  described  by  Ian  Buck 
and  Tim  Purcell  in  [Buck04].  This  is  an  example  of  the  requirement  of  tuning  an 
application  to  fit  the  constraints  of  GPU  processing. 

The  process  of  general-purpose  computing  on  the  GPU  begins  with 
loading  input  data  into  texture  memory.  Programs  are  also  loaded  onto  one  or 
both  of  the  programmable  segments  of  the  rendering  pipeline  to  perform 
calculations  on  the  pre-loaded  data.  Some  geometry  must  be  passed  to  the 
pipeline  to  trigger  processing  and  the  results  are  written  to  the  frame  buffer. 
(After  all,  the  programmable  portions  of  the  graphics  pipeline  will  not  be  activated 

if  no  primitives  are  being  processed!)  The  final  step  is  to  retrieve  the  information 
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from  the  frame  buffer  back  to  the  CPU  and  this  step  turns  out  to  be  surprisingly 
non-trivial. 

The  idea  of  passing  data  between  system  memory  and  the  graphics  card 
brings  us  to  our  most  limiting  and  often  most  important,  consideration  when 
considering  using  the  GPU  for  general-purpose  computing:  the  data  bus  between 
the  CPU  and  the  GPU.  The  only  distinction  between  the  Peripheral  Component 
Interconnect  (PCI),  the  Accelerated  Graphics  Port  (AGP),  and  the  new  PCI- 
Express  (PCI-X)  busses  is  the  magnitude  of  the  limitation.  Regardless  of  the 
data  bus,  the  exchange  of  data  between  the  CPU  and  the  graphics  card  will  be 
orders  of  magnitude  slower  than  the  exchange  between  either  processor  and  its 
memory.  With  this  in  mind,  it  is  easy  to  imagine  a  scenario  where  using  the  GPU 
is  less  efficient  than  simply  processing  the  data  on  the  CPU.  Generally,  any  time 
data  is  passed  to  the  GPU  for  processing,  enough  calculation  must  be  performed 
to  sufficiently  take  advantage  of  the  increased  performance  of  the  GPU  in  order 
to  offset  the  overhead  of  passing  inputs  and  outputs  through  the  CPU-GPU  data 
bus. 

4.  Summary 

Neither  the  GPU  nor  performing  general-purpose  calculations  on  it  are 
new  concepts.  More  and  more  studies  and  applications  are  being  developed  to 
exploit  the  performance  gap  between  GPUs  and  CPUs.  However,  despite  the 
increased  performance,  the  GPU  is  still  engineered  primarily  for  the  rendering  of 
geometric  data  to  a  display  of  some  sort.  Any  attempts  to  use  the  GPU  for 
general  calculations  must  consider  several  factors.  Not  the  least  of  which  are  the 
ideas  of  mandatory  parallelism  and  data  bus  overhead. 

D.  PIXEL  BUFFERS  (PBUFFERS) 

1.  Definition 

The  frame  buffer  is  the  default  destination  of  processed  geometry  to  be 
rendered  on  screen.  In  contrast,  a  pixel  buffer,  or  pbuffer,  is  a  destination  for 
geometry  that  is  not  destined  to  be  displayed,  at  least  not  directly.  A  pbuffer  is 
analogous  to  the  frame  buffer  on  many  levels.  Just  as  a  frame  buffer  is  a  portion 
of  memory  on  the  graphics  card,  a  pbuffer  is  also.  In  fact,  pbuffers  are 
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implemented  in  available  frame  buffer  memory.  Pbuffers  also  have  multiple 
rendering  buffers,  front-  and  back-left,  for  example,  a  depth  buffer,  and  often, 
stencil  and  accumulation  buffers.  These  are,  again,  similar  to  the  frame  buffer. 
The  difference  is  that  the  frame  buffer  can  be  mapped  directly  to  a  display  device 
whereas  the  data  in  a  pbuffer  must  first  be  transferred  to  the  frame  buffer  in  order 
to  be  displayed.  Logically,  since  pbuffers  are  non-visible,  they  are  commonly 
referred  to  as  being  off-screen  or  used  for  off-screen  rendering. 

2.  OpenGL  Specification 

The  OpenGL  specification  for  the  implementation  of  pbuffers  is  prescribed 
in  OpenGL  ARB  Extension  Number  11.  In  the  specification,  both  syntax  and 
semantics  are  described  in  addition  to  dependencies  on  other  OpenGL 
extensions.  Of  particular  note  is  a  recommendation  made  multiple  times  in  the 
specification:  “Pbuffers  should  be  deallocated  when  the  program  is  no  longer 
using  them  -  for  example,  if  the  program  is  iconified.”  [ARB1 1]  This  follows  from 
the  fact  that  pbuffers  take  resources  directly  from  the  frame  buffer.  On  the  other 
hand,  the  pbuffer  is  also  recommended  to  be  treated  as  a  relatively  static  entity. 
That  is,  it  should  not  be  reallocated  every  rendering  loop. 

3.  Synthetic  Vision  Impiementation 

Because  Synthetic  Vision  is  envisioned  to  work  as  part  of  a  visual 
simulation,  any  direct  intrusion  on  the  frame  buffer  would  be  visible  to  a  user. 
Not  only  would  the  user’s  apparent  frame  rate  be  effected,  but  her  overall 
experience  would  be  palled.  Pbuffers  provide  us  with  a  workspace  on  which  we 
can  manipulate  visual  information  without  distracting  the  user. 

Not  only  do  we  use  pbuffers  as  targets  to  render  off-screen,  but  also  as 
inputs  to  our  vertex  and  fragment  programs.  Several  ways  exist  to  access 
information  on  the  frame  buffer  and  pbuffers,  but  almost  all  involve  the  iterative 
reading  of  groups  of  pixels  from  texture  memory  and  copying  them  to  system 
memory.  These  methods  exacerbate  the  GPU-CPU  data  bus  issue  discussed 
earlier.  The  next  section  describes  an  alternative  method  of  retrieving  data  from 
the  pbuffer  without  the  exorbitant  overhead  generated  by  copying  pixels  across 
the  data  bus. 
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E.  RENDERING  TO  A  TEXTURE 

1.  Definition 

As  discussed  above,  a  texture  is  an  image  that  can  be  applied  to  the 
surface  of  a  set  of  primitives.  In  particular,  the  texture  is  stored,  appropriately,  in 
texture  memory  on  the  graphics  card  in  order  to  be  in  proximity  to  the  processor 
which  will  access  it,  probably  a  multitude  of  times.  The  idea  of  rendering  to  a 
texture  is  that  data  is  passed  to  the  graphics  card,  processed,  and  written  to  a 
pbuffer.  But  more  importantly,  that  image  on  the  pbuffer  is  now  directly 
accessible  to  the  GPU  in  the  same  manner  as  a  texture.  Without  rendering  to  a 
texture,  the  image  would  be  rendered  normally,  and  then  its  pixels  would  have  to 
be  copied  back  to  system  memory  through  the  data  bus  just  to  turn  around  and 
be  passed  back  to  the  graphics  card  to  be  stored  into  texture  memory. 
Rendering  to  a  texture  saves  a  round  trip  through  the  bus!  Depending  on  the 
size  of  the  texture,  the  savings  of  a  single  render  to  texture  could  be  in  the  order 
of  seconds,  not  milliseconds. 

2.  OpenGL  Specification 

The  OpenGL  specification  for  rendering  to  texture  is  prescribed  in 
OpenGL  ARB  Extension  Number  20.  The  extension  has  a  prefix  of  “WGL” 
indicating  that  it  is  only  specified  for  Microsoft  Windows  platforms.  Of  additional 
note  is  the  dependency  of  this  extension  on  the  pbuffer  extension  described 
above.  Several  constraints  must  be  addressed  when  using  this  extension.  Of 
central  concern  are  (from  [ARB20]): 

•  Only  color  buffers  of  a  pbuffer  can  be  bound  as  a  texture.  It  is  not 
possible  to  use  the  color  buffer  of  a  window  as  a  texture. 

•  When  a  color  buffer  of  a  pbuffer  is  being  used  as  a  texture,  the  pbuffer 
can  not  be  used  for  rendering;  this  makes  it  easier  for  implementations 
to  avoid  a  copy  of  the  image  since  the  semantics  of  the  pointer  swap 
are  clear. 
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•  The  application  must  release  the  color  buffer  from  the  texture  before  it 
can  render  to  the  pbuffer  again.  When  the  color  buffer  is  bound  as  a 
texture,  draw  and  read  operations  on  the  pbuffer  are  undefined. 

The  first  constraint  may  seem  intuitive.  We  find,  though,  that  rendering 
the  depth  values  of  a  system  of  geometry  can  be  useful,  too.  In  fact.  Synthetic 
Vision  uses  this  idea  to  make  depth  comparisons.  An  additional  extension 
enables  us  to  render  to  a  depth  texture.  It  is  WGL_NV_render_depth_texture 
[ARB263].  Again,  “WGL”  denotes  that  the  extension  is  only  specified  for 
Microsoft  Windows  and  the  additional  “NV”  similarly  denotes  nVidia  specificity.  In 
short,  ARB  Extension  Number  263  is  build  upon  the  ARB_render_texture 
extension  and  enables  a  pbuffer  to  be  bound  to  a  color  or  depth  texture. 

The  remaining  constraints  lay  out  a  rendering  cycle.  To  perform  any  non- 
visible  rendering,  the  pbuffer  must  first  be  enabled,  any  bound  color  texture  is 
released,  the  geometry  is  rendered  through  the  graphics  pipeline,  a  color  texture 
is  bound  to  the  newly-populated  pbuffer,  and  finally,  the  pbuffer  is  disabled  to 
allow  the  application  to  continue  on-screen  rendering. 

3.  Synthetic  Vision  Impiementation 

Rendering  to  texture  is  intrinsic  to  the  Synthetic  Vision  algorithm.  We 
render  two  images  to  textures  and  then  use  a  shader  program,  described  in  the 
next  section,  to  perform  pixel-by-pixel  comparisons  of  those  two  textures.  Again, 
without  the  ability  to  render  to  a  texture,  making  those  two  images  accessible  to 
the  shader  program  on  the  GPU  would  be  impossible  to  accomplish  in  a  timely 
fashion.  The  ability  to  generate  the  images  locally  on  the  graphics  card  and  then 
performing  calculations  on  those  images  with  the  performance  of  the  GPU  is  how 
this  algorithm  offsets  the  overhead  of  pushing  a  small  amount  of  data  across  the 
data  bus. 
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F.  SHADER  PROGRAMS 

1.  Definition 

Traditionally,  shader  programs  are  instruction  sets  defined  by  the 
application  which  specifically  effect  the  processing  that  occurs  at  the  two 
programmable  points  in  the  rendering  pipeline.  This  is  done  to  create  and  apply 
particular  effects  to  objects  in  the  rendered  scene.  Some  examples  provided  by 
[Rost04]  include: 

•  Increasingly  realistic  materials 

•  Increasingly  realistic  lighting  effects 

•  Non-photorealistic  effects 

•  Image  Processing 

The  two  programmable  points  mentioned  above  are  at  the  vertex 
processor  and  the  fragment  processor  and  the  particular  program  types  for  each 
point  are  appropriately  called  vertex  shaders  and  fragment  shaders.  One  or 
more  of  each  shader  type  can  be  compiled  and  linked  together  to  form  an 
executable.  A  shader  program  is  composed  of  one  or  more  of  these  executables 
and  is  run  on  the  programmable  vertex  and  fragment  processing  units. 
Syntactically,  the  two  programs  are  very  much  alike,  but  they  differ  in  their 
functions.  In  particular,  if  shader  programs  are  implemented,  they  must  perform 
the  same  functions  as  their  OpenGL  fixed  functionality  equivalent  in  addition  to 
whatever  special  behavior  the  shader  defines.  The  next  two  sections  will  discuss 
each  type  of  shader  in  slightly  more  detail, 
a.  Vertex  Shader 

In  fixed  functionality,  the  vertex  processor  performs  the  Geometry 
stage  functions.  That  is,  the  process  performs  the  transformations  of  vertices 
and  texture  coordinates  from  object  space  to  window  space.  Of  course,  much 
more  is  done,  but  these  are  the  functions  we  discussed  in  some  detail  earlier. 
Again,  the  vertex  shader  program  must  also  perform  these  functions.  These 
functions  are  fairly  straightforward  and  well  documented,  so  including  them  in  a 
custom  shader  is  no  great  ordeal. 
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What  the  vertex  shader  receives  as  inputs  are  built-in  variables 
such  as  color  and  position  which  are  passed  in  from  the  front  end  of  the 
rendering  pipeline.  User-defined  variables  and  textures  can  also  be  passed  into 
the  vertex  shader  from  the  application.  The  shader  performs  the  fixed-function- 
equivalent  functions  and  any  special  functions,  or  one  may  even  be  incorporated 
in  the  other.  Finally,  the  shader  returns  built-in  variables  such  as  position  and 
color  to  the  fixed  functionality  processes  in  place  between  the  vertex  and 
fragment  processors.  User-defined  variables  can  also  be  output  in  order  to  pass 
varying  data  from  the  vertex  shader  to  the  fragment  shader. 

Probably  most  importantly,  remember  that  the  programmable 
vertex  processor  does  just  as  the  name  implies  -  processes  vertices.  The  vertex 
program’s  calculations  are  only  performed  once  per  vertex.  Vertex  programs, 
generally,  lay  the  foundation  for  the  per-pixel  fragment  program  by  altering  the 
color  or  location  of  the  vertex  or  its  associated  texture  coordinate.  The  fragment 
processor  later  uses  these  perturbed  values  as  a  basis  for  whatever  effect  it  is 
designed  to  achieve. 

b.  Fragment  Shader 

Inputs  to  the  fragment  shader  come  from  three  sources:  built-in 
variables  passed  in  from  the  fixed  function  pipeline,  user-defined  variables  or 
textures  passed  in  from  the  application,  and  user-defined  inputs  which  were 
calculated  and  output  from  the  vertex  shader.  Care  must  be  taken  in  considering 
this  last  type  of  variable.  These  user-defined  variables  are  not  passed  directly 
from  the  vertex  processor  to  the  fragment  processor.  Remember,  the  input 
values  to  the  fragment  processor  are  interpolated  from  the  output  values  of  all 
the  vertices  of  the  primitive  to  which  the  fragment  belongs.  This  interpolation  is 
accomplished  by  the  section  of  fixed  function  between  the  two  processors. 

With  the  above  described  inputs,  the  fragment  shader  must,  just  as 
the  vertex  shader  does,  perform  calculations  to  fulfill  the  requirement  of  fixed- 
functionality  equivalence.  Texture  lookup  is  a  particularly  interesting  function 
performed  by  the  fragment  shader.  Normally,  given  a  texture  coordinate,  the 

fragment  shader  performs  a  lookup  on  the  input  texture  at  the  provided  texture 
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coordinate  and  retrieves  the  color  information.  Unlike  the  vertex  shader,  the 
fragment  shader  has  random  read  access  to  texture  memory.  That  is,  if  a  texture 
is  passed  in  as  in  input,  the  shader  can  read  the  color  value  of  any  pixel  in  that 
texture. 

c.  General-Purpose  Processing  Using  Shaders 

Consider,  again,  the  idea  that  fragment  shaders  can  perform 
random  look-ups  of  texture  memory.  This  is  the  means  by  which  sets  of  data  can 
efficiently  be  passed  to  the  GPU  for  general  purpose  calculations  -  stored  on  the 
video  card  in  the  format  of  a  texture!  If  the  data  is  correctly  partitioned  by  texture 
coordinates  and  accurately  addressed  by  the  vertex  shader,  multiple 
independent  calculations  can  be  performed  on  a  data  set  in  a  single  rendering 
pass  by  the  fragment  shader.  Of  course,  in  this  context,  rendering  pass  is 
somewhat  of  a  misnomer  in  that  no  rendering  is  really  being  accomplished.  In 
fact,  the  fixed  function  equivalence  that  we  are  normally  concerned  with  in 
creating  visual  effects  with  shader  programs  can  be  ignored  completely.  What 
cannot  be  ignored  is  the  requirement  to  assign  values  to  all  special  output 
variables  for  each  shader.  Most  of  these  outputs  will  be  used  as  designed  both 
by  the  vertex  and  fragment  shaders,  so  this  constraint  is  almost  trivial. 

While  texture  memory  is  used  to  load  input  data  to  the  GPU, 
vehicles  by  which  to  return  results  are  limited.  The  possible  options  all  revolve 
around  the  frame  buffer.  A  common  target  is  the  stencil  buffer.  Another  is  the 
frame  buffer  itself,  if  it  is  not  being  used  by  the  application  for  on-screen 
rendering.  Finally,  pbuffers  are  popular  targets  and  our  choice  for  returning 
results.  The  result  of  a  shader  program  is  then  a  pbuffer  populated  with, 
generally,  four  float  values  clamped  from  0  to  1.  In  itself,  a  rectilinear  array  of 
floats  is  unusable.  More  manipulations,  such  as  the  reduction  scheme 
mentioned  earlier,  must  be  performed  before  useful  results  can  be  gleaned. 

2.  Open  Graphics  Language  Shading  Language  (OGLSL) 

Individual  shader  code  very  much  resembles  assembly  language. 
However,  within  the  last  few  years,  higher-level  languages  have  been  developed 
to  make  the  development  of  shaders  more  resemble  the  workflow  in  developing 
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C  or  Java  code.  nVidia  and  Microsoft  formed  a  cooperative  partnership  of  sorts 
and  they  created  “C  for  Graphics”  (Cg)  and  the  “High  Level  Shading  Language” 
(HLSL),  respectively.  Meanwhile,  3DLabs  developed  a  C-like  language,  and 
supporting  OpenGL  extensions  encapsulating  the  functionality  of  shader 
programs,  named  OpenGL  Shading  Language  (OGLSL). 

In  June  2003,  3DLab’s  implementation  of  OGLSL  and  the  OpenGL 
extensions  that  support  it  were  adopted  as  OpenGL  ARB  extensions.  Further,  on 
3  September  2004,  the  newest  specification  of  OpenGL,  2.0,  was  officially 
approved.  Among  other  features,  OGLSL  and  its  APIs  were  added  to  the  core  of 
the  OpenGL  specification  [OpenGL].  If  graphics  card  vendors  continue  to  fully 
support  the  OpenGL  specification  as  in  the  past,  then  we  should  soon  see  full 
OGLSL  implementations  on  commodity  graphics  cards.  OGLSL  was  chosen 
over  Cg  and  HLSL  for  this  project  in  anticipation  of  its  acceptance  as  part  of  the 
core  of  OpenGL.  As  such,  we  expect  that  not  only  hardware  support  will 
continue,  but  the  technologies  described  above  should  also  continue  to  evolve, 
making  Synthetic  Vision  a  more  straightforward  and  accessible  architecture. 

3.  Synthetic  Vision  Impiementation 

Our  implementations  go  to  great  lengths  to  generate  and  bind  textures  as 
inputs  for  comparison  by  shading  programs.  As  it  turns  out,  once  these  input 
textures  -  whole  scene,  target-only,  and  their  depth  textures  -  are  in  place,  many 
different  comparison  schemes  are  possible.  Each  can  be  implemented  simply  by 
replacing  the  shader  which  performs  the  comparisons.  The  architecture  is  very 
modular  in  this  regard.  Also  possible  is  the  layering  of  multiple  different  shaders, 
each  will  be  executed  once  per  rendering  pass.  We  found  this  aspect  to  not  be 
optimal  for  real-time  simulation  applications,  but  offer  interesting  investigations 
involving  those  which  are  not  executed  at  real-time  or  near-real-time. 
Remember,  retrieving  results  from  the  shading  program  is  not  trivial.  After  the 
comparisons  are  complete.  Synthetic  Vision  implements  a  reduction,  distilling  a 
single  result  from  the  buffer  of  resultant  data.  This  reduction  is,  in  itself,  several 
successive  passes. 
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G.  SUMMARY 

The  only  way  to  tap  into  the  computing  power  of  the  GPU  is  currently 
through  shader  programs.  There  are  quite  a  few  restrictions  involving  not  only 
inputs  and  outputs,  but  also  the  access  of  either  by  the  CPU.  Despite  these 
restrictions,  the  speed  of  executing  these  simple  programs  can  absorb  the 
apparent  overhead  required  to  setup  the  problem  and  retrieve  the  results. 
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IV.  IMPLEMENTATION 


A.  INTRODUCTION 

Many  factors  played  roles  in  determining  the  path  to  implementing  our 
algorithm.  Of  course,  determining  feasibility  and  the  ability  to  reproduce  our 
results  were  considerations,  but  we  also  intended  to  engineer  the  additional 
ability  to  generalize  the  algorithms  to  be  able  to  reused  in  practical,  real-time 
applications.  In  the  end,  we  took  two  separate  roads:  a  relatively  small-scale 
‘proof-of-concept’  application  and  an  integrated  API-like  library.  SynVision  is  the 
application,  while  dtAI  is  an  integrated  library. 

B.  SYNVISION 

1.  Description 

While  rendering  to  texture  and  general-purpose  shaders  are,  separately, 
widely  known  and  used,  the  integration  of  the  two  is  much  less  so.  Examples  of 
each  exist,  but  the  amalgamation  of  these  examples  was  the  task  central  to 
creating  this  ‘proof-of-concept’  application.  The  particular  examples  and  the 
extent  to  which  they  were  used  are  listed  below  in  the  Architecture  subsection. 
Something  to  remember  is  that  SynVision  is  a  visualization  of  an  algorithm  that  is 
normally  invisible.  What  are  displayed  to  the  user  in  this  demo  are  processes 
that  would  be  non-visible  in  a  production  application. 

SynVision  is  a  simple  three-dimensional  world.  This  application 
represents  the  simplest  case  of  our  visibility  tests.  In  this  world  are  three  objects: 
a  background  object,  a  target  object,  and  an  obstacle  object  as  illustrated  in 
Figure  4.  Additionally,  the  point  of  view  can  be  moved  left  and  right,  giving  the 
user  the  ability  to  position  the  point  of  view  such  that  the  target  object  can  be  in 
the  open,  completely  obscured,  or  in  some  degree  of  partial  obscuration.  Along 
the  bottom  of  the  application  window  are  four  small  ‘mini-renders.’  These  are 
visualizations  of  the  textures  which  are  the  inputs  to  and  outputs  from  the 
comparison  shaders.  The  left-most  mini-render  is  the  whole-scene  texture.  To 
its  right  is  the  target-only  texture.  Continuing  to  the  right,  we  see  the  visible 
surface  of  the  target  against  a  red  background  and,  finally,  a  mini-render 
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visualizing  the  false  color  whole  scene  texture.  Statistics  including  the  number  of 
visible  pixels  can  also  be  displayed  in  the  console  window.  The  code  for 
SynVision  is  contained  in  Appendix  C. 


Figure  4.  SynVision  Demonstration  Application 

SynVision  currently  uses  two  separate,  but  related,  algorithms 
simultaneously:  color-based  visibility  and  false-color  visibility.  Color-based 
visibility  relies  on  comparing  two  textures,  each  in  their  ‘natural’  color,  while  false- 
color  visibility  makes  comparisons  of  the  colors  on  one  prepared  texture.  These 
differences  are  embodied  in  the  textures  used  as  inputs  to  the  comparison 
shaders.  The  algorithms  for  implementing  the  comparisons  in  the  main 
application  are  provided  in  Appendix  A,  while  the  specific  visibility  algorithms  are 
described  in  the  next  subsection. 
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2.  Visibility  Algorithms 

a.  Color-Based  Visibility  Algorithm 

The  primary  input  to  this  algorithm  is  the  texture  representing  the 
target  alone  against  a  clear  background  from  the  observer’s  viewpoint.  Another 
input  is  a  texture  of  the  whole  scene  just  as  it  would  be  seen  from  the  same 
viewpoint.  These  are  represented  by  panels  a  and  b,  respectively,  of  Figure  5. 


a.  Whole  Scene 


b.  Target-Only  c.  Visible  Surface 

Figure  5.  Color-Based  Visibility 
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Each  pixel  in  the  primary  texture  (b)  that  is  not  the  background 
color  is  compared  to  the  pixel  at  the  same  location  in  the  whole  scene  texture  (a). 
If  the  colors  are  the  same,  then  that  pixel  is  considered  to  be  visible  and  it  is 
written  to  the  visible  surface  texture  (c).  Otherwise,  it  is  not.  Panel  c  of  Figure  5 
illustrates  the  result  of  this  comparison. 

The  color-based  visibility  scheme  is  simply  a  binary  algorithm  and 
is  not  sensitive  to  color  mutations  such  as  the  effects  of  smoke  or  partially 
transparent  surfaces,  for  instance,  those  used  to  model  screen  doors.  Visible 
pixel  counts  can  also  be  erroneously  high  if  the  target  is  obscured  by  similarly- 
colored  object. 

b.  False-Color  Visibility  Algorithm 


Figure  6.  False  Color  Visiblity 


The  False-Color  algorithm  is  slightly  more  complex  to  set  up  than 
its  Color-Based  equivalent,  but  simpler  to  make  actual  comparisons.  It  is  an 
adaptation  of  an  occlusion  culling  algorithm  presented  in  [KlimOS].  When 
rendering  the  whole  scene,  a  shader  is  used  to  create  a  flat  scene  composed  of 

three  colors:  background  color,  target  color,  and  non-target  color.  What  is 
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meant  by  “flat  scene”  is  one  which  has  no  apparent  depth;  everything  in  the 
scene  is  drawn  as  a  monochromatic  silhouette  with  no  details  or  shading.  This 
results  in  a  two-dimensional  image  in  which  all  pixels  composing  the  target  are  a 
single  color,  and  everything  else  is  another.  Figure  6  shows  the  results  of  a  color 
scheme  where  the  target  is  white,  the  background  is  black,  and  all  other  objects 
are  gray. 

The  comparison  shader  has  a  relatively  simple  job,  then  -  given  the 
color  assigned  to  all  target  pixels  (white,  in  the  above  Figure)  as  an  input, 
compare  every  pixel’s  color  to  that  input  color.  If  the  pixel  is  ‘target-colored’,  then 
it  is  considered  to  be  visible.  Since  all  models’  color  information  is  discarded  in 
encoding  with  the  false  colors,  this  algorithm  is  also  not  sensitive  to  semi-opaque 
surfaces. 

3.  Architecture  and  Components 

a.  OpenGL 

SynVision  is,  at  its  heart,  an  OpenGL  application.  The  run  cycle  is 
implemented  using  a  standard  library,  GL  Utility  Toolkit  (GLUT).  This  cycle  is 
essentially  the  same  loop  as  discussed  above  in  the  Chapter  III.  GLUT  manages 
the  execution  of  this  loop  by  requiring  the  application  to  register  functions  with 
each  stage  of  the  cycle.  For  example,  void  display  (void)  is  the  signature 
of  the  routine  that  is  called  at  the  ‘draw’  stage  of  each  frame.  GLUT  also 
encapsulates  the  creation  and  modification  of  windows  in  which  to  render. 

In  using  OpenGL,  extensions  are  often  used  to  access 
functionalities  of  a  graphics  card  that  are  not  available  widely  enough  to  be 
accepted  as  core  OpenGL  components.  While  these  extensions  are  not  a  part  of 
the  core  of  OpenGL,  they  are  still  standards  with  specified  behaviors  and  syntax. 
For  example,  SynVision  must  use  the  Microsoft  Windows-only  extensions 
WGL_ARB_PBUFFER  and  WGL_ARB_RENDER_TEXTURE,  to  name  a  few. 
To  manage  these  extensions  and  to  be  able  to  test  if  they  are  available  at  run¬ 
time,  the  GL  Extension  Wrangler  (GLEW)  library  is  used.  While  OpenGL  2.0  has 
recently  been  released,  this  application  was  developed  on  OpenGL  Version  1 .5. 
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b.  Shaders 

An  overview  of  the  OpenGL  Shader  Language  is  given  above  in 


Chapter  III.F.2.  It  is  used  in  SynVision  to  implement  the  comparison  and 
reduction  functions  as  described  briefly  in  Table  1.  Our  methods  of  reading 
shader  source,  creating  instances  of  shader  programs,  and  linking  the  fragment 
shaders  to  those  programs  are  based  loosely  on  an  example  from  [Rost04], 
named  ogl2brick.  The  source  code  for  this  example  can  be  found  at  [RostWeb], 
the  companion  website  to  [Rost04]. 


Name 

Scheme 

Function 

boolvis.frag 

Color-Based 

Crop  all  pixels  not  belonging  to  Target 

colorbasedvis.frag 

Color-Based 

Encode  pixels  as  visible  or  not  visible 

boolvis2.frag 

False  Color 

Color  pixels  as  visible  or  not  visible 

sumreduce.frag 

Reduction 

Perform  Sum  over  entire  pbuffer 

Table  1.  SynVision  Shaders 


Both,  the  visibility  comparison  and  the  reduction  shaders  are  fairly 
simple  programs.  Through  the  methods  described  above,  the  programs  are 
read,  created,  linked,  and  compiled.  While  SynVision  currently  has  two  sets  of 
shaders,  this  configuration  is  not  immutable.  Any  interesting  visibility  comparison 
scheme  can  be  implemented  by  a  new  shader  if  the  required  inputs  can  be 
bound  to  that  shader.  This  new  shader  can  complement  or  replace  the  existing 
shaders.  In  short,  the  shaders  are  very  modular.  Normally,  with  small 
infrastructure  modifications  to  bind  the  correct  inputs,  comparison  schemes  in  the 
form  of  shader  programs  can  be  easily  swapped, 
c.  RenderTexture 

RenderTexture.cpp  and  its  associated  header  file, 
RenderTexture. h,  are  integral  to  SynVision.  RenderTexture  is  a  Component  that 
encapsulates  the  render  to  texture  process  to  include  the  instantiation  of  a 
pbuffer  and  the  associated  textures  to  be  bound  to  the  pbuffer,  to  include  depth 
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textures.  RenderTexture  allows  us  way  to  implement  the  multiple  renders  to 
textures  required  by  our  algorithm  a  relatively  intuitive  way.  It  provides  us  a 
means  to,  first,  instantiate  a  pbuffer  and  its  associated  textures,  and  then  to 
enable  and  disable  the  render-to-texture  process.  Additionally,  methods  are 
provided  to  customize  the  pbuffer  and  textures  and  to  query  the  states  of  those 
objects  at  run-time. 

RenderTexture  was  originally  created  and  is  copyrighted  by  Mark 
Harris;  but,  it  is  openly  licensed  with  a  few  caveats.  The  details  of  the  licensing 
can  be  found  in  the  header  of  either  rendertexture.cpp  or  rendertexture.h. 
RenderTexture  and  Harris’  other  real-time  graphics  research  can  be  found  at 
[HarrisWeb]. 

d.  Modeling  Components 

The  models  used  in  SynVision  were  created  using  MilkShape  3D. 

Several  files  are  required  to  load  and  draw  these  models.  These  files  are: 

o  MilkshapeModel.h 
o  MilkshapeModel.cpp 
o  Model. h 
o  Model. cpp 
o  ModelUtil.cpp 

All  but  the  last  file,  ModeUtil.cpp,  are  from  NeHe  Tutorial  Number 
31  [NeHe31].  They  were  created  and  copyrighted  by  Brett  Porter.  The  last  file  is 
a  utility  file  that  loads  a  texture  from  a  file  into  texture  memory.  Together,  these 
files  make  the  conversion  from  a  MilkShape  3D  file  to  a  set  of  primitives  and  their 
associated  textures. 

C.  DTAI:  PROPOSED  SCENEGRAPH  IMPLEMENTATION 
1.  Description 

Our  desired  path  to  reusability  and  generality  involves  implementing  the 
Synthetic  Vision  algorithms  in  a  simulation  architecture.  Currently  at  the  Naval 
Postgraduate  School’s  MOVES  Institute,  a  simulation  engine  is  being  developed. 
We  will  discuss  Delta3D  in  more  detail  below  in  subsection  2.b.  dtAI 
(pronounced  “delta  Al”)  is  a  proposed  component  of  the  Delta3D  engine. 
(Components  of  Delta3D  have  ‘dt’  suffixes,  hence  the  Al  component  is  dtAI) 
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While  the  implementation  of  Synthetic  Vision  was  the  impetus  of  dtAI,  it 
turned  out  to  be  only  one  facet.  dtAI  can  be  considered  to  be  the  fledgling 
Artificial  Intelligence  component  of  DeltaSD.  This  component  is  desired  to 
encapsulate  the  required  functionality  of  computer  generated  forces,  whether 
teammates  or  opposing  forces  and  to  our  knowledge,  it  will  be  one  of  very  few 
open  sourced  artificial  intelligence  APIs. 

In  order  to  implement  Synthetic  Vision  as  part  of  dtAI,  a  number  of 
architectural  components  were  first  developed.  These  components  include  a 
representation  of  the  synthetic  player,  AlObject,  and  a  manager  of  these  players, 
AlObjectManager.  Additionally,  implementations  of  a  pbuffer,  and  mechanisms 
for  rendering  to  texture  and  implementing  shader  programs  were  also  put  into 
place.  In  dtAI,  these  components  are  PBuffer,  RenderToTextureStage,  and 
Viewinterpreter,  respectively. 

Currently,  all  of  these  components  above  are  implemented  in  DeltaSD  and 
stand  as  a  foundation  for  further  development  of  both  a  usable  artificial 
intelligence  API  and  a  test  bed  for  research,  such  as  Synthetic  Vision  itself.  In 
the  next  section,  we  describe  dtAI’s  underlying  architectures.  Appendix  B 
describes,  in  detail,  the  components  of  dtAI. 

2.  Architecture  and  Components 

a.  The  Open  Scene  Graph  (OSG) 

i.  Description.  The  Open  Scene  Graph  is  a  cross¬ 
platform,  open  source  application  toolkit  written  in  C++  and  OpenGL  [OSG1].  It 
is  widely  supported  by  a  public  community  and  is  actively  developed.  It  started  in 
1998  as  a  project  by  Don  Burns.  In  1999,  Robert  Osfield  joined  Burns  in 
developing  the  OSG.  That  same  year,  the  source  code  was  open  sourced  and 
Robert  took  on  the  role  of  project  lead.  Burns  and  Osfield  continue  to  drive  the 
development  of  the  OSG  and  also  support  users  of  the  OSG  through  their  private 
companies.  [OSG2]  The  OSG  is  open  source,  licensed  with  the 
OpenSceneGraph  Public  License  which  is  based  on  the  GNU  Library  Public 
License. 
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ii.  Scene  Graph.  To  understand  the  OSG,  we  must  first 
discuss  scene  graphs.  Simply,  a  scene  graph  is  a  tree  of  nodes.  Each  node 
represents  an  object  in  the  scene.  Each  node  also  has  attributes  and  these 
attributes  are  specific  to  each  type  of  node.  Some  examples  of  nodes  are 
drawable  objects,  transformable  (movable)  objects,  and  lights.  The  notion  of  a 
tree  in  the  sense  of  data  structures  also  implies  that  a  node  can  be  the  parent  of 
child  nodes.  In  the  case  of  transformables,  when  moving  the  parent  node  to  a 
new  position,  the  child  nodes  are,  normally,  also  moved  based  on  some  offset 
relative  to  the  parent.  This  is  an  example  of  one  of  the  simpler  relationships 
present  in  a  scene  graph.  The  idea  of  the  scene  graph  begins  simply,  but  as 
complexity  and  robustness  are  added  and  further  compounded,  the  structure  can 
quickly  become  unwieldy.  This  is  where  the  OSG  comes  in  -  the  OSG  defines 
the  types  of  nodes,  manages  the  relationships,  and  most  importantly,  maintains 
structural  and  syntactic  standards  throughout  the  scene  graph. 

One  reason  that  the  scene  graph  is  organized  as  a 
tree  is  that  an  operation  central  to  the  scene  graph  is  a  traversal.  The  OSG’s  run 
cycle  is  significantly  more  complex  than  that  of  GLUT.  Understanding  traversals 
is  a  key  to  understanding  the  OSG’s  run  cycle.  Instead  of  a  simple  update-and- 
draw  cycle,  the  OSG  performs  update,  cull,  and  draw  traversals.  On  each 
traversal,  if  a  node  is  flagged  as  having  work  to  be  done  during  that  traversal,  the 
work  is  executed.  For  example,  some  nodes  on  the  scene  graph  are  not 
drawable  and  so  nothing  is  done  by  that  node  during  the  draw  traversal. 

The  way  a  node  flags  that  it  has  work  to  be  done 
during  a  traversal  is  through  callbacks.  If  a  node  desires  to  execute  some 
process  during  each  cull  traversal,  for  example,  it  creates  a  cull  traversal  callback 
which  will  execute  that  process  once  per  cull  traversal.  There  are  usually  also 
pre-  and  post-  traversal  callbacks  which  are  executed,  appropriately,  before  and 
after  the  simple  callback. 

iii.  Dependencies.  The  OSG  is  principally  built  upon  two 
other  libraries:  OpenThreads  and  Open  Producer.  OpenThreads  is  an  open 
source  library  designed  to  manage  multi-threaded  applications  and  Open 
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Producer  is  a  programming  interface  which  encapsulates  the  actual  rendering  of 
the  information  represented  by  the  scene  graph.  Open  Producer  operates  on  the 
analogy  of  a  camera  complete  with  lens  (point-of-view  frustum)  and  film  (render 
surface).  Open  Producer  was  also  developed  and  is  currently  maintained  by  Don 
Burns’  consulting  business. 

iv.  Summary.  The  OSG  is  a  robust,  open  source 
graphics  library  composed  of  a  scene  graph  coupled  with  the  rendering 
capabilities  of  Open  Producer.  It  is  portable,  free,  and  still  actively  developed. 
Traversals  are  central  to  performing  operations  over  the  entire  scene  graph  and 
callbacks  are  the  methods  by  which  work  is  done  by  a  node  during  each 
traversal. 

b.  Delta3D  Simulation  Engine 

i.  Description.  The  Modeling,  Virtual  Environments,  and 
Simulation  (MOVES)  Institute  is  located  at  the  Naval  Postgraduate  School  in 
Monterey,  California.  One  working  group  at  the  Institute  is  developing  a  portable, 
high-performance  simulation  engine  [Delta].  The  project,  managed  by  Erik 
Johnson,  is  on  a  course  to  engineer  a  library  with  which  high-fidelity,  highly- 
immersive  simulation  applications  can  be  designed  and  implemented  relatively 
quickly.  This  library  brings  together  numerous  input  devices  from  mouse-and- 
keyboard  to  inertial  trackers  and  output  devices  such  as  head-mounted  displays 
and  CAVE  projection  systems.  The  final  product  is  an  applications  library 
tailored  to  networked  simulation. 

ii.  Architecture.  The  OSG  is  at  the  heart  of  DeltaSD.  It  is 
the  model  of  any  application  written  using  DeltaSD.  OSG,  at  its  core,  is  a  data 
structure  and  a  means  to  display  that  data.  It  is  not  directly  designed  to  referee 
physical  collisions  or  perform  character  animations,  for  example.  So  DeltaSD 
integrates  the  OSG  with  a  few  other  libraries  such  as  Open  Dynamics  Engine 
(ODE),  Character  Animation  Library  (CALSD),  and  others.  With  these  functional 
components  integrated  into  a  single  application  programming  interface,  DeltaSD 
has  the  potential  to  be  a  robust  applications  development  library  freely  available 
via  the  GNU  Library  Public  License  [Delta]. 
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iii.  Summary.  DeltaSD  is  a  portable,  open  source 
simulation  engine.  dtAI  is  proposed  as  a  component  to  it.  As  a  component,  dtAI 
is  more  a  complement  to  DeltaSD  than  a  user  of  that  API.  Except  at  the  topmost 
levels,  dtAI  is  primarily  a  user  of  the  OSG.  In  other  words.  Synthetic  Vision  is  a 
functionality  developed  using  the  OSG  and  is  encapsulated  by  dtAI  for  use  with 
DeltaSD. 

3.  Visibility  Algorithm 

The  algorithm  for  determining  visibility  in  dtAI  is  an  extension  of  the  Color- 
Based  visibility  algorithm  discussed  in  section  B.2.a.  It  adds  the  ability  to  perform 
depth  comparisons  on  each  pixel  as  well  as  color.  Armed  with  this  additional 
information,  we  can  partially  address  the  weaknesses  mentioned  in  both  tested 
visibility  algorithms.  That  is,  we  can  now  make  assumptions  concerning 
situations  which  affect  a  color  change  in  the  target  object,  like  shadowing,  by 
comparing  both  the  depth  and  color  of  the  pixel  in  the  whole  scene  compared  to 
the  pixel  at  the  same  position  in  the  target-only  scene.  In  particular,  if  the  depth 
values  are  the  same  but  the  color  values  are  different,  then  the  surface  color  of 
the  target  has  changed.  The  specific  algorithm  is  proposed  as  follows: 

For  each  pixel  comprising  the  target  in  the  Target-Only  image: 

•  If  the  depth  of  this  pixel  in  the  Target-Only  image  is  the  same  as  the 
pixel  at  this  location  in  the  Whole-Scene  image,  then: 

o  If  the  color  of  this  pixel  is  the  same  in  both  images  then: 

■  Label  this  pixel  as  ‘Visible’ 

o  Else,  the  color  is  different  from  one  image  to  the  other.  Label  this 
pixel  as  ‘Hazy’  and  encode  the  differences  in  color. 

•  Else,  a  pixel  of  another  object  is  obscuring  the  target  pixel.  If  the  alpha 
value  is  not  1.0,  signifying  semi-transparence,  then: 

o  Label  this  pixel  as  ‘Hazy’  and  encode  the  differences  in  color. 

•  Else,  label  this  pixel  as  ‘Not  Visible’ 


43 


This  additional  functionality  comes  at  a  price.  The  price  is  relatively  small 
in  computational  terms,  but  much  larger  in  an  architectural  sense.  This  nearly 
doubles  the  complexity  of  the  pbuffer  to  add  a  second  texture  and  the  methods  to 
initialize,  bind,  release,  and  destroy  them.  There  is  also  an  increase  in  the 
number  of  OpenGL  extensions  required,  further  limiting  the  domain  of  computers 
able  to  execute  Synthetic  Vision.  Currently,  with  the  use  of  pbuffers  and  render- 
to-texture,  Microsoft  Windows  is  the  required  platform.  With  the  addition  of 
rendering  to  depth  texture,  these  platforms  are  additionally  reduced  to  those 
which  support  the  ARB_NV_RENDER_DEPTH_TEXTURE  extension.  As  an 
_NV_  extension,  only  nVidia  graphics  cards  can  be  assumed  to  support  this 
extension.  Once  implemented,  an  investigation  should  be  made  to  determine  if 
the  ability  to  detect  color  changes  allay  the  additional  overhead  and  platform 
limitations. 

D.  SUMMARY 

SynVision  is  a  demonstration  application.  It  is  the  visualization  of  what  is 
designed  to  be  computed  off-screen,  invisible  to  the  user.  More  importantly,  it  is 
the  “proof  of  concept”  that  pbuffers  can  be  used  as  inputs  to  comparison 
functions  in  the  form  of  OGLSL  shader  programs,  enabling  execution  of  Synthetic 
Vision  to  be  performed  by  the  GPU  and  its  associated  memory.  Two  different 
visibility  algorithms  are  implemented  in  SynVision:  Color-Based  and  False  Color 
comparison  schemes.  By  using  the  two  different  algorithms,  a  convenient 
modularity  was  exposed.  It  is  simple  to  use  either  one  or  both  of  the  algorithms 
by  simply  disabling  small  sections  of  code.  SynVision  could  easily  be  modified  to 
allow  the  enabling  and  disabling  of  one  or  both  algorithms  at  run-time. 

dtAI  is  a  proposed  component  of  the  MOVES  Institute’s  DeltaSD 
Simulation  Engine.  Much  work  was  done  towards  reaching  this  goal,  but  further 
work  must  be  completed  before  dtAI,  and  thus  a  completely  integrated 

implementation  of  Synthetic  Vision,  can  be  realized.  dtAI  can  currently  be 
compiled  against  DeltaSD,  dated  14  September  as  downloaded  from  DeltaSD’s 
CVS  server.  As  dtAI  is  continued  to  be  developed  and  eventually  integrated  into 
DeltaSD,  the  latest  version  will  be  available  online  at 
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http://www.cupertinist.net/dtAI/index.html.  Current  issues,  bugs, 
recommendations  for  investigation  will  also  be  available. 


and 
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V.  TESTING  AND  RESULTS 


A.  INTRODUCTION 

As  discussed  in  the  Introduction  in  Chapter  1,  determining  the  visibility  of  a 
target  within  a  synthetic  player’s  field  of  view  is  currently  achieved  by  determining 
if  a  line-of-sight  exists  between  the  player  and  the  target.  More  specifically,  line 
of  sight  is  normally  determined  by  some  implementation  of  ray  casting.  The 
principle  reason  this  method  is  used  is  that  the  algorithms  are  well  known,  and 
more  importantly,  they  are  well  known  to  execute  quickly.  Any  proposed 
alternative  to  casting  a  ray  in  determining  a  target’s  visibility  must  also  execute 
quickly  enough  to  not  decrease  the  performance  of  the  overall  application. 

The  demonstration  application,  SynVision,  was  used  to  determine  the 
timing  efficiency  of  the  visibility  algorithms.  The  successful  implementation  of 
SynVision,  itself,  answers  the  basic  question  of  whether  the  algorithms  can  be 
implemented  using  the  GPU.  With  that  in  mind,  we  concentrated  on  testing  the 
more  quantifiable  aspect  -  algorithm  execution  times.  In  building  the 
demonstration,  we  also  delved  briefly  into  the  realm  of  visibility  algorithms.  As 
described  in  the  last  chapter  in  detail,  we  successfully  implemented  two  separate 
visibility  schemes.  In  developing  those  schemes,  we  gained  some  insight  into 
some  basic  strengths  and  limitations  of  our  architecture. 

The  next  section  briefly  describes  our  test  platform.  The  subsequent 
sections  will  outline  the  testing  process  that  was  used  to  determine  the  timing 
performance  of  each  algorithm  and  the  overall  application.  Finally,  we  will 
summarize  the  findings  of  these  tests  by  pairing  our  results  with  the  research 
questions  posed  in  Chapter  1 . 

B.  TEST  SYSTEM 

SynVision  was  tested  on  a  laptop  computer  equipped  with  a  3.06GHz  Intel 
Pentium  4  processor,  512MB  of  DDR  RAM,  and  an  nVidia  FX  Go5600  video 
chipset  with  128MB  of  dedicated  RAM.  Iwas  written,  debugged,  and  tested  in  the 
integrated  development  environment,  Microsoft  Visual  Studio  2003. 
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C.  METHODOLOGY 

The  Windows. h  API  was  used  to  calculate  the  execution  times. 
QueryPerformanceCounter(),  coupled  with  QueryPerformanceFrequency()  gave 
a  much  finer  time  resolution  than  the  clock()  or  getTime()  methods  included  in 
time.h.  This  resolution  was  1/3059250000,  or  3.2688x10'^°,  seconds  as 
compared  to  clock()’s  1/1000  second  resolution. 

To  time  a  section  of  code,  the  system  time  was  first  queried  immediately 
before  the  code  was  executed.  This  pre-time  was  assumed  to  have  no 
significant  overhead  associated.  Another  system  time  was  queried  at  the 
completion  of  the  section  of  code.  The  time  associated  with  making  this  query  is 
considered  to  be  overhead  and  is  calculated  once  per  frame.  The  overhead  was, 
on  average,  3.25x10'^  seconds.  The  resulting  time  is  the  difference  of  the  pre- 
and  post-times  minus  the  overhead.  Determining  the  frame  time,  or  the  amount 
of  time  required  to  perform  one  complete  run  cycle,  was  found  by  querying  the 
time  immediately  prior  to  executing  the  run  cycle  and  then  once  per  frame,  at  the 
completion  of  the  draw  cycle.  This  resulting  time  was,  again,  the  difference 
between  the  “current”  time  and  “previous”  times  minus  the  overhead  value.  For 
all  tests,  the  calculated  times  per  frame  were  accumulated  for  seven  minutes  and 
averaged  over  the  number  of  frames  which  were  executed  in  those  seven 
minutes. 

The  results  of  a  single  timing  calculation  came  in  the  form  of  "’ticks.”  A 
tick  is  defined  differently  for  each  computer.  For  our  testbed,  a  tick  was 
consistently  1/3059250000  seconds.  We  say  consistently  because,gg  for  some 
laptops  and  handheld  devices  which  scale  processing  in  order  to  manage  power 
consumption,  the  frequency  of  the  ticks  can  be  dynamic.  Our  test  platform  did 
not  vary  the  tick  frequency. 

To  generate  average  times,  we  had  to  accumulate  these  single  timing 
calculations.  We  did  this  by  converting  the  single  timing  to  seconds  and  adding  it 
to  the  current  count.  We  also  accumulated  the  number  of  frames  which  had 
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occurred.  To  find  the  average  at  the  test’s  end,  the  accumulated  time  was 
divided  by  the  number  of  frames.  This  result  was  now  in  seconds. 

D.  RESULTS 

In  executing  the  time  testing,  we  found  that  the  times  to  perform  the 
renders  to  texture  were  far  overshadowed  by  the  time  required  to  perform  the 
summing  reductions  of  each  visibility  algorithm.  For  the  sake  of  comparison,  the 
rendering  times  are  listed  in  Table  2.  Each  render  is  listed  with  the  algorithm  in 
which  it  is  employed  and  its  associated  execution  time.  Of  note,  the  large 
disparity  between  the  Color-Based  and  False  Color  Whole  Scene  renders  can  be 
attributed  to  the  fact  that  the  False  Color  render  is  done  by  executing  “flat 
drawing.”  That  is,  no  lighting  effects  or  textures  are  applied  to  the  models,  every 
pixel  in  the  model  is  simply  drawn  a  single  color.  Similarly,  the  Figure-Only 
render  is  a  single  model  against  a  clear  background,  but  took  relatively  long  to 
render  due  to  the  application  of  lighting  and  textures. 


Render 

Algorithm 

Time  in  Milliseconds 

Whole  Scene 

Color-Based 

1.14740 

Figure-Cniy 

Color-Based 

0.69530 

Visible  Surface  Cniy 

Color-Based 

0.61708 

Encoding 

Color-Based 

0.55607 

Whole  Scene 

False  Color 

0.45748 

Encoding 

False  Color 

0.67410 

Table  2.  Rendering  Times  in  Milliseconds 


For  each  algorithm,  two  times  were  taken:  the  frame  time  and  the  time  to 
distill  the  results,  whether  by  counting  or  reducing  and  summing.  Table  3 
provides  these  results.  Frame  Time  is  the  time  required  for  the  main  application 
to  complete  a  single  run  cycle,  or  frame.  Reduction  Time  is  that  amount  of  time 
required  to  perform  either  the  summing  reduction,  or  counting,  of  the  comparison 
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shader’s  resultant  buffer.  While  Buffer  Count  is  not  an  algorithm,  it  is  included  in 
the  results  for  comparison.  The  Frame  Time  associated  with  Buffer  Count  is 
actually  the  time  required  to  execute  the  False  Color  algorithm  and  instead  of 
performing  a  summing  reduction  on  the  results  from  the  encoding  shader,  it 
simply  iterates  through  all  the  pixels  in  those  same  results  and  counts  the 
number  of  visible  pixels.  The  time  to  perform  just  this  iteration  is  Counting  Time. 


Algorithm 

Component 

Time  in  mSec 

Frames/Second 

Color-Based 

Frame  Time 

23.4692 

42 

Reduction  Time 

21.3793 

False  Color 

Frame  Time 

18.5348 

53 

Reduction  Time 

17.1425 

Buffer  Count 

Frame  Time 

21.4497 

46 

(False  Color) 

Counting  Time 

20.4674 

Table  3. 

Frame  Rates  and  Times  in  Milliseconds  by  Al 

gorithm 

E.  SUMMARY  OF  FINDINGS 

1.  Considerations 

First  and  foremost,  while  SynVision  models  a  very  simple  three- 
dimensional  world,  it  is  not  an  optimized  application.  It  is,  for  the  study’s  benefit, 
very  straightforward  and  iterative.  Secondly,  testing  of  the  algorithms  was 
performed  using  SynVision  in  the  debugging  mode  of  the  IDE.  Hence,  the 
application  was  not  stripped  and  optimized  by  the  compiler.  Both  of  these  facts 
point  to  results  that  should  be  considered  fairly  conservative. 

The  times  that  were  found  in  our  tests  reflect  results  in  applying  the 
algorithms  to  a  single  window  resolution  of  512x512  pixels.  This  resolution  maps 
directly  to  the  size  of  the  pbuffers  used.  Changing  resolutions  directly  affects  the 
amount  of  the  finite  frame  buffer  resources  available.  Perhaps  more  importantly, 
the  size  of  the  pbuffers  has  an  immediate  impact  on  the  reductions  used  to  sum 
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the  shader  results.  Comparing  resolutions  of  512x512  and  1024x1024,  only  one 
more  pass  -  eight  versus  seven  -  is  required  to  complete  a  reduction  of  the 
larger  pbuffer,  while  the  number  of  iterations  in  a  counting  scheme  would 
increase  by  a  factor  of  four  -  1048576  versus  262144.  We  did  not  test  this 
scenario,  but  it  is  easy  to  conjecture  that  the  closeness  of  times  between 
counting  and  executing  a  reduction  at  512x512  would  not  exist  at  the  larger 
resolution.  As  we  implied  earlier,  the  results  of  the  time  testing  should  be 
considered  somewhat  conservative. 

2.  Research  Questions 

a.  Feasibility  of  Impiementing  the  Architecture  Without 
Using  the  Programmabiiity  of  the  GPU 

If  we  assume  that  pbuffers  and  rendering  to  texture  are  supported 
by  the  video  card,  then  a  visibility  test  centered  on  the  false  color  algorithm  can 
be  implemented.  The  Buffer  Count  entries  in  Table  2  are  results  of  that  very 
implementation.  The  color-based  algorithm  can,  in  theory,  also  be  implemented. 
Textures  are,  after  all,  simply  multidimensional  arrays.  The  comparisons  could 
be  painstakingly  done  between  two  textures,  and  the  encoding  would  involve 
writing  to  a  third,  very  large,  array  to  create  the  resultant  texture.  Neither  of 
these  requirements  seems  able  to  be  done  in  a  computationally-efficient  manner, 
so  no  attempt  to  implement  this  without  using  shaders  was  attempted. 
Remember,  an  alternative  to  ray  casting  must  offer  an  increase  of  performance 
or  accuracy  or,  as  our  intention,  both. 

b.  Feasibility  of  Impiementing  the  Algorithms  Using  the 
GPU 

The  demonstration  application,  SynVision,  attests  to  the  feasibility 
of  implementing  these  algorithms  using  the  programmability  of  the  GPU.  While 
not  optimal,  it  still  provides  the  number  of  visible  pixels  for  a  target  at  real-time 
speeds.  The  successful  implementation  of  both  the  color-based  and  the  false 
color  algorithms  in  SynVision  is  encouraging.  Many  avenues  are  now  open  to 
further  research  from  developing  more  visibility  algorithms  to  optimizing  the 
architecture.  Our  recommendations  for  future  work  are  in  the  final  chapter. 
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c.  Possible  Algorithms  for  Making  Comparisons 

Color-Based  and  False  Color  are  the  two  visibility  algorithms 
implemented  by  SynVision.  A  third  algorithm  proposed  for  use  in  dtAI  adds  the 
depth  information  to  mitigate  the  effects  of  obscurations.  The  development  of 
additional  algorithms  is  an  on-going  challenge.  This  study  did  not  broach  the 
subject  of  computer  vision  and  the  numerous,  well-known  and  implemented 
algorithms  such  as  color  or  texture  histograms.  Initial  consideration  points  to 
investigating  those  computer  vision  algorithms  that  use,  as  inputs,  values  that 
can  be  taken  directly  from  the  graphics  pipeline  such  as  depth,  color,  and 
opacity. 

At  first  look,  the  linchpin  in  determining  the  efficiency  of  these 
visibility  algorithms  seems  to  be  the  number  of  reductions  that  are  required. 
Optimally,  this  reduction  operation  is  only  done  once  to  gather  the  final  results. 
Perhaps  refinement  of  the  reduction  process’s  implementation  will  lessen  the 
strength  of  that  statement,  but  investigations  in  that  direction  are  left  for  future 
endeavors. 
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VI.  CONCLUSIONS 


A.  INTRODUCTION 

Measuring  the  ability  to  detect  an  object  in  a  three-dimensional  simulation 
by  determining  whether  an  observer  has  line  of  sight  to  a  target  is  reasonable, 
even  intuitive.  The  problem  lies  in  implementing  a  way  to  determining  if  line  of 
sight  exists.  The  current  methodologies  to  evaluate  line  of  sight  are,  generally, 
ray  casting  schemes.  In  three-dimensional  worlds,  ray  casting  is  a  geometric 
solution  to  a  perceptual  problem.  In  implementing  ray  casting,  either  too  much  or 
too  little  information  can  be  given  to  the  synthetic  player. 

In  this  study,  we  proposed  an  alternative  to  ray  casting.  Synthetic  Vision. 
By  using  the  information  inherent  to  a  rendered  scene,  a  target’s  visibility  can  be 
determined.  Further,  the  access  and  computing  power  of  the  programmable 
graphics  processing  unit  can  be  leveraged  to  execute  this  scheme. 

Two  implementations  of  Synthetic  Vision  were  intended:  a  demonstrative 
visualization,  SynVision,  and  a  library  to  be  added  to  the  DetlaSD  simulation 
engine,  dtAI.  SynVision  was  created  and  was  used  to  test  two  separate 
detection  algorithms.  dtAI  has  not  been  fully  developed;  but,  has  been  designed 
as  outlined  in  Appendix  B.  Following  are  the  conclusions  drawn  from  these 
components  and  some  recommendations  for  future  work. 

B.  CONCLUSIONS 

1.  SynVision 

SynVision  demonstrates  that  employing  the  GPU  in  determining  the 
visibility  of  targets  in  a  synthetic  player’s  field  of  view  is  certainly  possible.  We 
suggested  that  different  visibility  detection  algorithms  should  be  able  to  be  used. 
Two  different  algorithms  were  implemented:  Color-Based  Comparison  and  False 
Color  Comparison.  SynVision  was  also  used  to  employ  the  False  Color 
algorithm  without  using  shaders.  This  implementation  was  about  15  percent 
slower  than  the  same  algorithm  using  shaders. 
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Not  only  does  SynVision  confirm  the  ability  to  implement  Synthetic  Vision, 
but  it  also  helped  to  realize  the  variability  in  the  different  possible  visibility 
comparison  algorithms.  While  the  frame  rates  and  rendering  times  of  both 
implementations  of  SynVision  can  be  considered  real  time  (>30  frames  per 
second),  the  False  Color  algorithm  was  consistently  quicker  than  its  Color-Based 
equivalent.  On  the  topic  of  frame  rates,  SynVision’s  performance  is  encouraging, 
especially  considering  it  is  an  application  whose  strengths  lie  in  visualization,  not 
speed. 

2.  dtAI 

A  guiding  goal  of  this  study  was  to  have  an  implementation  of  Synthetic 
Vision  that  could  be  used  in  a  real-time,  three-dimensional  simulation.  A  library 
to  be  used  as  part  of  the  DeltaSD  Simulation  Engine,  dtAI,  is  our  bid  towards  that 
end.  This  vision  was  not  fully  realized.  This  is  due  primarily  to  an 
underestimation  of  the  integration  effort  required  to  implement  Synthetic  Vision  in 
DeltaSD’s  scene  graph,  the  Open  Scene  Graph.  The  OSG  has  facilities  for  using 
shaders  and  also  for  off-screen  rendering,  but  both  are  engineered  to  eventually 
be  displayed  to  the  user.  To  derail  that  process  required  much  more 
infrastructure  than  we  originally  anticipated. 

Although  the  implementation  was  not  completed,  its  development  is  on¬ 
going.  With  the  encouraging  success  of  SynVision,  we  still  expect  the  integration 
of  Synthetic  Vision  into  the  OSG  to  be  a  useful  addition  to  DeltaSD.  With  this 
successful  integration  will  come  a  whole  new  level  of  opportunity  to  develop  and 
test  real-time  visibility  algorithms.  As  mentioned  above,  progress  towards  this 
goal  will  be  documented  at  http://www.cupertinist.net/dtAI. 

3.  Synthetic  Vision 

Garnering  visibility  information  from  images  created  by  rendering  a  scene 
from  a  synthetic  player’s  view  point  and  using  that  information  to  determine 
whether  that  player  should  ‘see’  an  object  within  its  field  of  view  is  Synthetic 
Vision.  It  has  been  realized  using  commodity  graphics  hardware  and  has  been 
shown  to  execute  at  real  or,  at  least,  near-real  time.  While  the  performance  of 
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SynVision  is  promising,  overall  performance  of  Synthetic  Vision  will  depend  on 
both  the  implementation  and  the  visibility  algorithm  utilized. 

C.  RECOMMENDATIONS  FOR  FUTURE  RESEARCH 
1.  Optimizations 

a.  Reduction  Process 

A  single  value  must  be  distilled  from  the  buffer  of  values  resulting 
from  the  execution  of  a  shader.  In  SynVision,  we  implemented  a  reduction 
strategy  which  took  a  512x512  buffer  and  quartered  that  area  each  pass  until  the 
area  was  2x2.  We  then  summed  the  elements  of  that  2x2  array.  We  also 
investigated  a  brute-force  counting  scheme  which  would  iterate  through  the 
262144  elements  of  the  resultant  buffer.  At  our  test  resolution,  the  counting 
scheme  was  only  about  15  percent  slower  than  the  reduction  strategy. 

We  hypothesize  that  a  combination  of  reduction  and  counting  would 
outperform  either  scheme  executed  separately.  For  example,  instead  of 
reducing  the  512x512  to  2x2  and  summing  those  elements,  perhaps  we  reduce 
to  64x64  and  count  those  4096  elements  iteratively.  We  expect  that  for  each 
initial  resolution,  there  is  an  optimal  mix  of  reduction  and  counting  to  perform  this 
distillation. 

b.  OpenGL  Context  Switching 

One  of  the  most  computationally  expensive  operations  in  the 
OpenGL  API  is  a  context  switch.  OpenGL  is  a  state-based  system.  A  context  is 
a  complete  set  of  OpenGL  state  attributes.  Switching  these  contexts  requires  the 
unloading  of  one  complete  state  and  loading  another.  Context  switching  is  a 
factor  in  our  algorithm  because  each  pbuffer  has  an  individual  context.  We  use 
multiple  pbuffers  and  each  time  we  either  enable  or  disable  a  pbuffer,  a  context 
switch  occurs.  This  equates  to  multiple  context  switches  per  frame.  Further 
research  into  the  minimizing  of  this  context  switching  would  directly  affect  the 
performance  of  Synthetic  Vision.  In  particular,  investigation  of  context  sharing, 
as  in  wgIShareListsO,  or  a  reuse  scheme  are  reasonable  initial  directions. 
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c.  SynVision  System  Design 

As  mentioned  earlier,  SynVision  is  not  currently  optimized  for 
speed.  It  was  simply  a  workspace  for  assembling  the  architectures  required  to 
visualize  Synthetic  Vision.  While  originally  a  “proof  of  concept”  application, 
SynVision  was  found  to  be  useful  for  visualizing  the  various  visibility  algorithms. 
Optimizing  SynVision  towards  this  task  of  visualization  will  result  in  the  creation 
of  an  aid  in  the  development  of  visibility  algorithms. 

One  proposed  method  to  realize  a  performance  gain  in  SynVision 
involves  OpenGL  optimization.  In  addition  to  reducing  the  number  of  context 
switches  as  described  above,  minimizing  the  number  of  state  changes  is  a 
standard  means  of  optimizing  OpenGL  performance. 

Another  performance  gain  could  be  realized  by  SynVision  by 
refining  the  architecture  of  the  reduction  method.  In  particular,  SynVision  uses 
the  frame  buffer  and  a  copy  to  texture  to  execute  the  reduction.  By  using  two 
pbuffers  which  share  a  graphics  context,  the  computationally  expensive  copy  to 
texture  can  be  avoided. 

2.  Visibility  Algorithms 

Two  visibility  algorithms  were  investigated  using  SynVision:  Color-Based 
and  False  Color.  We  used  these  two  in  order  to  validate  the  demonstration  by 
calculating  the  number  of  visible  pixels  by  two  different  methods  and  comparing 
the  results.  Despite  the  difference  in  the  algorithms,  the  results  matched 
consistently.  Many  algorithms  already  exist  in  the  field  of  Computer  Vision  to 
make  comparisons  of  regions  of  an  image.  We  mentioned  that  Synthetic  Vision 
addresses  problems  somewhat  reversed  from  those  in  Computer  Vision,  but 
there  seems  to  be  a  sizeable  overlap  in  the  algorithms  that  can  be  used  to  solve 
both.  An  optimized  SynVision  could  be  a  good  platform  for  developing  and 
testing  these  algorithms  and  a  functional  dtAI  would  certainly  be  a  fair  test  bed. 
In  either  case,  we  expect  many  algorithms  for  determining  visibility  can  be  taken 
from  Computer  Vision  and  other  fields  and  implemented  in  Synthetic  Vision. 
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3.  Implementation  in  Constructive  Simulations  Augmented  by 
Three-Dimensional  Model  Information 

This  concept  is  the  furthest  from  fruition  and  would  require  a  great  shift  in 
the  way  constructive  simulation  are  built  and  marketed.  In  general,  constructive 
simulations  are  composed  of  mathematical  models,  to  include  the  terrain.  The 
interface  is  often  a  top-down,  bird’s  eye  view  of  the  battlespace  overlaid  onto  a 
map  or  some  other  two-dimensional  representation  of  the  terrain.  Line  of  sight  is 
just  as  important  in  these  simulations  as  they  are  in  virtual,  three-dimensional 
simulations.  Targeting,  engagements,  and  communications  are  examples  of 
processes  heavily  reliant  on  line  of  sight. 

If,  in  addition  to  the  mathematical  models  of  each  object  on  the  battlefield, 
there  existed  graphical  models.  Synthetic  Vision  could  be  employed  to  determine 
line  of  sight  reliably,  accurately,  and  off  the  already  busy  CPU.  The  attributes  of 
these  graphical  models  such  as  orientation  and  position  are  already  implemented 
in  the  constructive  simulation.  A  quick  off-screen  render  could  be  done  from  the 
observer  to  the  target  to  determine  the  visibility  of  the  target.  Sun  and  moon 
position  would  be  directly  modeled,  as  would  the  target’s  posture.  Instead  of 
using  constants  interpolated  from  tables  to  find  the  single  value  representing  the 
visibility  of  all  soldiers  kneeling  in  the  open  with  overhead  sunlight,  a  run  through 
Synthetic  Vision  would  give  a  dynamic  result.  That  result  would  be  appropriately 
different  from  the  result  of  evaluating  the  same  target  from  a  nearby  observer’s 
vantage  point. 
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APPENDIX  A:  SYNVISION  APPLICATION  ALGORITHM 


//  Initialization 

Initialize  Application  Window 

Ensure  OGLSL  Extensions  are  Supported  by  Hardware 
Initialize  and  Position  Camera  Representing  the  Point  of  View 
Initialize  the  Models:  Background,  Obstacle,  and  Target 
Initialize  Pbuffers:  Whole  Scene,  Target,  Visualization,  and  Comparison 
(Includes  Initializing  Textures  to  be  Bound  to  Pbuffers) 

Initialize  Shaders:  Visualization,  Comparison,  and  Reduction 

Loop  Until  Termination: 

//  Render  entire  scene  to  the  Whole  Scene  Texture 
Enable  Whole  Scene  Pbuffer 
Draw  All  Models 

Bind  Whole  Scene  Texture  to  Pbuffer 
Disable  Whole  Scene  Pbuffer 

//  Render  Target  against  a  cleared  background  to  the  Target  Texture 

Enable  Target  Pbuffer 

Draw  Target  Model 

Bind  Target  Texture  to  Pbuffer 

Disable  Target  Pbuffer 

//  Render  an  image  of  the  visible  portion  of  the  Target  against 
//  a  clear  background  to  the  Visualization  Texture 
Enable  Visualization  Pbuffer 

Set  Whole  Scene  Texture  as  an  Input  to  Visualization  Shader 
Set  Target  Texture  as  an  Input  to  Visualization  Shader 
Set  Additional  Variables  as  Inputs  as  Required 
Enable  Visualization  Shader  (Algorithm  Described  Below) 

Draw  a  Single  Quad  Spanning  the  Entire  Viewport 
Disable  Shader 

Bind  Visualization  Texture  to  Pbuffer 
Disable  Visualization  Pbuffer 
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//  Generate  a  Boolean  map  of  visible  and  non-visible  Target  pixels 
Enable  Comparison  Pbuffer 

Set  Whole  Scene  Texture  as  an  Input  to  Comparison  Shader 
Set  Target  Texture  as  an  Input  to  Comparison  Shader 
Set  Additional  Variables  as  Inputs  as  Required 
Enable  Comparison  Shader 

Draw  a  Single  Quad  Spanning  the  Entire  Viewport 
Disable  Shader 

Perform  Summing  Reduction  on  Comparison  Pbuffer 
Display  Results  to  Console  Window 
Disable  Comparison  Pbuffer 

//  Render  the  Whole  Scene,  but  encode  the  pixels  of  the  target  one  color 
//  and  everything  else  another  color 
Enable  Whole  Scene  False  Color  Pbuffer 

Draw  Background  and  Obstacle  Models  All  One  Color 
Draw  Target  Model  A  Different  Color 
Bind  False  Color  Texture  to  Pbuffer 
Disable  Whole  Scene  False  Color  Pbuffer 

//  Generate  Boolean  map  of  visible  and  non-visible  pixels 
Enable  False  Color  Comparison  Pbuffer 

Set  Whole  Scene  False  Color  Texture  as  an  Input  to  Comparison  Shader 
Set  Target  Color  as  an  Input  to  Comparison  Shader 
Enable  Comparison  Shader 

Draw  a  Single  Quad  Spanning  the  Entire  Viewport 
Disable  Shader 

Perform  Summing  Reduction  on  Comparison  Pbuffer 
Display  Results  to  Console  Window 
Disable  False  Color  Comparison  Pbuffer 

//  Render  the  User’s  View  (the  “real”  scene  plus  each  of  the  rendered 
//  textures) 

Draw  All  Models  in  the  “Main”  Window 

Draw  Quads  with  the  Whole  Scene,  Target,  and  Visualization  Textures 
//  Update 

Read  any  Inputs  from  User 

Update  Camera  Position  as  Required,  Given  Inputs 

End  Loop 
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APPENDIX  B:  DTAI  CLASSES 


AlObiect 

AlObject  is  simply  a  computer  generated  entity  and  as  such,  an  object  in  the 
scene.  In  particular,  it  is  an  dtCore::Object  that  can  be  drawn,  transformed 
(moved),  and  acted  upon  physically.  AlObject  can  be  a  friend  or  foe  of  the 
human  player,  or  both.  The  methods,  Decide()  and  Act()  are  called  in  an  update 
traversal  callback.  As  an  Object,  methods  for  moving  AlObject  are  inherited  from 
dtCore: Transformable,  and  physical  traits  from  dtCore;:Physical. 

Members: 

Focus.  A  Point  in  world  coordinates  at  which  the  AlObject  is  looking 
Eye  Position.  A  point  in  world  coordinates  from  which  AlObject  observes 
Eye  Offset.  A  vector  in  object  space  from  the  AlObject’s  model’s  origin  to 
the  Eye  Position 

Target.  An  object  on  whose  origin  AlObject  is  focused 
List  of  Targets  A  list  of  objects  in  AlObject’s  field  of  view  which  have  been 
categorized  at  least  as  Detected 

Field  of  View.  A  geometric  frustum  representing  AlObject’s  field  of  view. 
Methods: 

Decide.  Process  inputs  and  choose  an  action. 

Act.  Update  AlObject  to  implement  Decide’s  choice. 

Getters  and  Setters  for  each  Attribute  listed  above. 


AlObiectManager 

AlObjectManager  provides  target  detection,  intra-team  adjudication  and 
communication  between  teams  of  AlObjects.  Currently,  AlObjectManager  only 
implements  the  Synthetic  Vision  algorithm  for  target  detection.  AlObjectManager 
is  a  non-drawable  node  on  the  scene  graph.  It  is  iterated  through  on  each 
traversal,  but  is  not  drawn  on  the  draw  traversal.  Target  detection  is  performed 
for  all  AlObjects  in  the  AlObjectManager’s  team  each  frame  in  a  cull  traversal 
callback.  We  chose  the  cull  traversal  because  the  output  of  performing  target 
detection  should  be  made  available  at  the  update  traversal.  Traversals  are  multi¬ 
threaded,  so  if  target  detection  was  executed  during  the  update  traversal,  not  all 
targets  would  be  processed  where  they  actually  appear  after  the  traversal. 
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Members: 

List  of  team  members  (AlObjects) 

Node.  Pointer  to  AlObjectManager’s  osg::Node  on  the  scene  graph 

Eye  Point.  A  single  ‘camera’  to  be  used  when  traversing  through  team 
members  to  perform  target  detection.  This  camera  is  moved  to 
each  AlObject’s  eye  position  and  Synthetic  Vision  is  performed 
from  that  point  of  view. 


Methods: 

Add  Team  Member 
Remove  Team  Member 
Get  List  of  Team  Members 

Process  Targets.  Execution  of  Synthetic  Vision  Algorithm  on  each  team 
member. 


PBuffer 

This  is  the  implementation  of  the  pixel  buffer  as  described  in  Chapter  3.  It  can 
not  be  created  until  the  application’s  windowing  system  creates  graphics  and 
device  contexts.  From  these  contexts,  OpenGL  derives  the  pbuffer’s  contexts. 

Members: 

Last  Graphics  and  Device  Contexts.  Before  making  the  pbuffer  current, 
store  these  values  so  we  can  restore  these  contexts  later 

PBuffer.  Actual  OpenGL  pixel  buffer 

Pbuffer  Contexts.  Pbuffer’s  Graphics  and  Device  Contexts 

Size 

Draw  Buffer.  Current  portion  of  the  pbuffer  being  rendered  to 
(WGL_FRONT_LEFT_ARB,  for  example) 

Initialization  Booleans.  Values  required  to  create  the  pbuffer.  Examples 
are  doubleBuffered,  RGB,  shareLists. 

Pixel  Format  Variables.  Also  necessary  to  create  the  pbuffer.  These  are 
generally  minimum  number  of  bits  required  for  each  type  of  buffer 
(depth,  alpha,  stencil,  etc) 

State  Booleans.  Run-time  queries  concerning  the  state  of  the  pbuffer 
Methods: 

Handle  Mode  Switch.  Ensures  the  pbuffer  is  never  lost  without  being 
recreated 

Create  pbuffer.  Initialization  routine 

Enable  pbuffer.  Makes  the  pbuffer  the  current  rendering  context 

Disable  pbuffer..  Makes  the  Last  Contexts  current  again. 

Bind  Textures.  Bind  textures  to  the  pbuffer’s  frame  and  depth  buffers 

Release  Textures.  Releases  the  textures.  This  must  be  done  before 
rendering  to  the  pbuffer.  Otherwise,  results  are  undefined. 

Setters  and  Getters  for  some  Members 
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RenderToTextureStage 

In  the  OSG,  a  RenderStage  is  an  encapsulation  of  a  complete  stage  in  rendering. 
“Stage”  is  used  here  in  the  Hollywood  sense  -  lights,  camera,  and  only  those 
objects  that  are  supposed  to  be  seen  from  the  given  camera  angle. 
RenderToTextureStage  is  a  derived  class  of  osgUtil::RenderStage  that  renders 
to  a  pbuffer  instead  of  the  frame  buffer.  RenderToTextureStage  encapsulates 
pbuffer  and  owns  the  textures  which  are  to  be  bound  to  that  pbuffer. 
dtAI::RenderToTextureStage  should  not  be  confused  with 
osgUtil::RenderToTextureStage  which  copies  the  texture  from  the  frame  buffer  to 
the  CPU  back  to  texture  memory. 

Members: 

PBuffer 

Size 

Color  and  Depth  Textures.  These  textures  will  be  bound  to  the  pbuffer. 
Render  Mode.  Choice  between  Depth  Texture  or  Color  Texture  or  both. 

Methods: 

Draw.  Overriden  method  of  osgUtil::RenderStage.  It  enables  the  pbuffer 
before  doing  the  standard  draw,  binds  the  textures  to  the  pbuffer 
and  then  diables  the  pbuffer. 

Set  Viewport.  Overriden  method  of  osgUtil::RenderStage.  If  the  size  of 
the  Viewport  has  changed,  resize  the  pbuffer  as  well. 

Initialize  Textures 

Getters  and  Setters  for  the  Members 


OffscreenSceneView 

An  osgUtil::SceneView,  to  a  large  extent,  performs  the  OSG’s  run  cycle. 
Generally,  there  is  one  SceneView  per  camera  in  the  scene.  This  run  cycle  has 
three  basic  steps:  Update,  Cull,  and  Draw.  OffscreenSceneView  is  derived  from 
SceneView  for  the  purpose  of  rendering  off-screen  (to  the  pbuffer).  In  order  to 
render  off-screen  and  not  update  the  scene  graph,  the  three  stages  are 
overridden. 

Methods: 

Update.  Does  nothing  since  this  cycle  is  executed  for  each  AlObject, 
each  frame.  We’ll  let  the  main  application  perform  Updates. 

Cull.  Given  a  node  from  the  scene  graph,  decide  which  of  the  node  and 
its  children  should  be  on  stage. 

Draw.  Draw  those  objects  which  are  on  stage. 
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Viewinterpreter 

Viewl interpreter  is  the  shader  utility  portion  of  the  library.  It  specifically  creates, 
intitializes,  and  updates  the  shaders  used  to  do  visibility  comparisons  and  also  to 
perform  the  summing  reductions.  It  is  taken  in  large  part  from  an  OSG  example, 
osgshaders.  In  addition  to  the  shaders,  it  also  creates  the  geometry  over  which 
the  shaders  will  be  executed.  In  theory,  this  is  a  quad  that  is  the  same  size  as 
the  viewport.  This  is  most  easily  accomplished  by  setting  the 
OffscreenSceneView  to  be  an  orthographic  projection  aligned  with  the  quad. 

Members: 

Root  Node.  The  osg::Group  which  is  the  root  of  the  quad  to  be  rendered 
Whole  Scene  Color  Texture.  These  textures  are  used  as 
Whole  Scene  Depth  Texture.  uniform  variable  inputs 
Target  Color  Texture.  to  the  visibility 

Target  Depth  Texture.  comparison  shaders 

View  Program  Object.  An  OGLSL  program  consisting  of  a  single  fragment 
shader  to  be  used  for  visibility  comparisons 
View  Fragment  Program.  Visibility  Comparison  shader 
Reduce  Program  Object.  An  OGLSL  program  to  perform  reductions 
Reduce  Fragment  Program.  Summing  Reduction  shader 

Methods: 

Load  Shader  Source.  Retrieve  the  shaders’  source  code  from  a  file 
Enable  Shader 

Set  Textures.  Set  the  textures  to  be  used  as  inputs  to  shaders 
Initialize  Shaders.  Creates,  links,  and  compiles  shaders. 

Build  Scene.  Initialize  the  quad  on  which  the  shaders  will  be  run 


OffscreenSceneHandler 

A  Producer::Camera::SceneHandler  is  an  abstract  class  which  prepares  the 
scene  to  be  rendered  for  the  Producer: :Camera.  OffscreenSceneHandler  is  a 
class  derived  from  SceneHandler  and  is  taken  largely  from 
dtCore::Scene::_SceneHandler.  After  all,  the  off-screen  rendering  must  exactly 
mimic  the  on-screen  rendering,  especially  with  regard  to  lighting.  In  fact,  a 
consideration  that  has  not  been  implemented  in  dtAI  is  the  synchronization  of  the 
application’s  SceneHandler  with  OffscreenSceneHandler.  They  both  have  the 
same  defaults,  but  if  SceneHandler  is  changed,  then  the  off-screen  render  no 
longer  mimics  the  on-screen  one. 
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Members: 

OffscreenSceneView.  This  is  the  link  to  the  camera. 

List  of  Possible  Targets.  After  culling  an  AlObject’s  point  of  view,  the 
custom  CullVisitor  produces  this  list  of  possible  targets. 

Current  RenderToTextureStage.  Three  RenderToTextureStages  are  used 
in  this  algorithm.  This  is  the  one  being  used  currently. 

Custom  CullVisitor.  This  culls  the  AlObject’s  point  of  view  and  collects 
pointers  to  the  osg::Nodes  of  all  the  objects  within  the  field  of  view. 
This  is  used  by  the  whole  scene  RenderToTextureStage 

Default  CullVisitor.  This  culls  the  point  of  view,  but  does  not  produce  a  list 
of  possible  targets. 


Methods: 

Clear.  Resets  the  Members 

Cull.  Collect  all  objects  in  the  AlObject’s  field  of  view  for  drawing 

Draw.  Draw  the  objects  collected  by  Cull 

Frame.  Called  by  Camera  to  initiate  the  run  cycle 

Set  Scene  Data.  Set  an  osg::Node  as  the  head  of  a  subsection  of  the 
scene  graph.  All  culling  and  drawing  will  be  done  on  this  node  and 
its  children  and  below  on  the  scene  graph. 

Set  Draw  Buffer.  Set  the  buffer  of  the  pbuffer  to  which  the  scene  will  be 
rendered 

Set  RenderToTextureStage.  Set  the  current  RenderToTextureStage 
Member 

Set  CullVisitor.  Set  the  active  CullVisitor.  Custom  for  the  whole  scene, 
and  default  for  all  others. 


AlEyepoint 

This  is  the  Synthetic  Vision  workhorse.  This  combines  a  Producer::Camera,  a 
OffscreenSceneHandler  and  OffscreenSceneView,  a  Viewinterpreter,  and  three 
RenderToTextureStages  to  implement  the  visibility  algorithm.  This  brings  it  all 
together. 

Members: 

Viewinterpreter.  Encapsulates  the  shaders 

Root  Node.  The  current  head  of  the  subsection  of  the  scene  graph  to  do 
work  on.  To  process  the  whole  scene,  the  Root  Node  is  the  head 
of  the  application’s  entire  scene  graph.  To  process  a  target,  the 
Root  Node  is  simply  the  target’s  osg::Node. 

Camera.  Producer::Camera  which  is  moved  to  each  AlObject’s  Eye 
Position.  From  this  position,  that  AlObject’s  point  of  view  is 
rendered  and  processed. 

OffscreenSceneHandler.  Processes  each  RenderToTextureStage  at  each 
step  of  the  algorithm. 
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Whole  Scene  RenderToTextureStage.  Produces  the  texture  containing 
the  entire  scene  from  the  AlObject’s  point  of  view. 

Target  RenderToTexture  Stage.  Produces  the  texture  containing  a  single 
target  against  a  clear  background  from  the  AlObject’s  point  of  view. 

Results  RenderToTextureStage.  Produces  a  texture  encoded  with  the 
visilibility  of  the  target  at  each  pixel.  This  is  also  used  in  the 
Reduction  process. 


Methods: 

Process  Field  of  View.  Execute  the  Synthetic  Vision  algorithm. 
Initialize.  Initialize  the  RenderToTextureStages  and  shaders. 
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APPENDIX  C 


SYNVISION.CPP 


#include  "RenderTexture . h" 


#include 

#include 

#include 

#include 


<GL/ glut . h> 
<Windows . h> 
<assert . h> 
<stdio . h> 


#include  "Model. h" 

#include  "MilkshapeModel . h" 


unsigned  char  *buf; 
GLdouble  eyeX; 


Model  *pModelGround  =  NULL;  //  Holds  The  Model  Data 
Model  *pModelFigure  =  NULL;  //  Holds  The  Model  Data 


void  Reshape (int  w,  int  h) ; 

GLuint  iTextureProgram  =  0; 

GLuint  iPassThroughProgram  =  0; 

//RenderTexture  *rt  =  NULL; 
RenderTexture  *rtFig  =  NULL; 
RenderTexture  *rtAll  =  NULL; 
RenderTexture  *rtVis  =  NULL; 
RenderTexture  *rtRed  =  NULL; 
RenderTexture  *rtA112  =  NULL; 
RenderTexture  *rtRed2  =  NULL; 


GLhandleARB  visTexProg; 

GLhandleARB  visBoolProg; 

GLhandleARB  reduceProg; 

GLhandleARB  visBool2Prog; 

bool  bShowDepthTexture  =  false; 

LARGE_INTEGER  lastFrame; 
int  ticks  =  0; 

double  accumFrameTime  =  0.; 

double  accumReductionlTime  =  0.; 
double  accuinReduction2Time  =  0.; 
double  accumCountTime  =  0.; 

int  printOglError (char  *file,  int  line); 

#define  printOpenGLError ( )  printOglError (  FILE  ,  LINE  ) 
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// - 

//  Function  :  printInfoLog 

//  Description  : 

/ /  From  ogl2brick; 

// - 

void  printInfoLog (GLhandleARB  ob j ) 

{ 

int  inf ologLength  =  0; 
int  charsWritten  =  0; 

GLcharARB  *infoLog; 

printOpenGLError ( ) ;  //  Check  for  OpenGL  errors 

glGetObjectParameterivARB (obj ,  GL_OBJECT_INFO_LOG_LENGTH_ARB, 

&inf ologLength) ; 

printOpenGLError 0 ;  //  Check  for  OpenGL  errors 

if  ( inf ologLength  >  0) 

{ 

infoLog  =  (GLcharARB* ) malloc ( inf ologLength) ; 
if  (infoLog  ==  NULL) 

{ 

printf  ( "ERROR:  Could  not  allocate  InfoLog  bufferin''); 
exit ( 1 ) ; 

} 

glGetlnfoLogARB (obj ,  inf ologLength,  ScharsWritten,  infoLog); 
printf ("InfoLog: \n%s\n\n",  infoLog) ; 
free ( infoLog) ; 

} 

printOpenGLError ( ) ; 

} 


// - 

//  Function  :  isExtensionsSupported 

//  Description  : 

/ /  From  ogl2brick 

//  The  recommended  technique  for  querying  OpenGL  extensions; 

//  from  http://opengl.org/resources/features/OGLextensions/ 

// - 

int  isExtensionSupported (  const  char  *extension  ) 

{ 

const  GLubyte  *extensions  =  NULL; 
const  GLubyte  *start; 

GLubyte  *where,  *terminator; 

/*  Extension  names  should  not  have  spaces.  */ 
where  =  (GLubyte  *)  strchr (extension,  '  '); 

if  (where  | |  *extension  ==  '\0') 
return  0; 

extensions  =  glGetString (GL  EXTENSIONS); 

/*  It  takes  a  bit  of  care  to  be  fool-proof  about  parsing  the 
OpenGL  extensions  string.  Don't  be  fooled  by  sub-strings,  etc.  */ 
start  =  extensions; 
for  (;;)  { 
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where  =  (GLubyte  *)  strstr ( (const  char  *)  start,  extension); 
if  ( ! where) 
break; 

terminator  =  where  +  strlen (extension) ; 
if  (where  ==  start  | |  * (where  -  1)  ==  '  ') 

if  (*terminator  ==  '  '  | I  *terminator  ==  '\0') 

return  1; 

start  =  terminator; 

} 

return  0; 

} 


// - 

//  Function  :  shaderSize 

//  Description  : 

//  From  ogl2brick  (modified) 

//  Returns  the  size  in  bytes  of  the  shader  fileName. 

//  If  an  error  occurred,  it  returns  -1. 

// - 

int  shaderSize (char  *fileName) 

{ 

int  shader; 
int  count; 

//  Open  the  file 

shader  =  lopen (fileName,  OF  READ); 
if  (shader  ==  -1) 
return  -1; 

//  Seek  to  the  end  and  find  its  position 
count  =  llseek ( shader ,  0,  SEEK  END); 

_lclose (shader) ; 
return  count; 

} 


// - 

//  Function  :  readShader 

//  Description: 

//  Reads  a  shader  from  the  supplied  file  and  returns  the  shader  in  the 
//  arrays  passed  in.  Returns  1  if  successful,  0  if  an  error  occurred. 

//  The  parameter  size  is  an  upper  limit  of  the  amount  of  bytes  to  read. 

//  It  is  ok  for  it  to  be  too  big. 

// 

//  From  ogl2brick  (modified) 

// - 

int  readShader (char  *fileName,  char  *shaderText,  int  size) 

{ 

FILE  *shader; 
int  count; 

//  Open  the  file 

shader  =  fopen (fileName,  "r"); 

if  ( ! shader) 

return  -1; 
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//  Get  the  shader  from  a  file, 
f seek ( shader ,  0,  SEEK  SET); 

count  =  fread (shaderText,  1,  size,  shader); 
shaderText [count]  =  '\0'; 

if  ( ferror ( shader ) ) 
count  =  0; 
else  count  =  1; 

fclose (shader) ; 
return  count; 

} 


// - 

//  Function  :  readShaderSource 

//  Description  : 

/ /  From  ogl2brick 

// - 

int  readShaderSource (char  *fileName,  GLcharARB  **fragmentShader) 

{ 

int  fSize; 

//  Allocate  memory  to  hold  the  source  of  our  shaders. 
fSize  =  shaderSize (fileName) ; 

if  (fSize  ==  -1) 

{ 

printf ( "Cannot  determine  size  of  the  shader  %s\n",  fileName); 
return  0; 

} 

*fragmentShader  =  (GLcharARB  *)  malloc (fSize) ; 

//  Read  the  source  code 

if  (! readShader (fileName,  *fragmentShader,  fSize))  { 

printf ( "Cannot  read  the  file  %s.frag\n",  fileName); 
return  0; 

} 

return  1; 

} 
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:  installShader 


// - 

//  Function 
//  Description 
/ /  From  ogl2brick; 

// - 

GLhandleARB  installShader (GLcharARB  *fragment) 

{ 

GLhandleARB  shader,  program;  //  handles  to  objects 
GLint  f ragCompiled;  //  status  values 

GLint  linked; 

//  Create  a  fragment  shader  object 

shader  =  glCreateShaderOb j ectARB (GL_FRAGMENT_SHADER_ARB) ; 

//  Load  source  code  strings  into  shaders 

glShaderSourceARB ( shader ,  1,  (const  char** ) Sfragment,  NULL); 

//  Compile  the  brick  vertex  shader,  and  print  out 

//  the  compiler  log  file. 

glCompileShaderARB ( shader ) ; 

printOpenGLError ( ) ; 

glGetOb j  ectParameterivARB ( shader, 

GL_OBJECT_COMPILE_STATUS_ARB,  &f ragCompiled) ; 
printInfoLog ( shader ) ; 

if  (! f ragCompiled) 
return  0; 

//  Create  a  program  object  and  attach  the  two  compiled  shaders 
program  =  glCreateProgramObj ectARB () ; 
glAttachObj ectARB (program,  shader) ; 

//  Link  the  program  object  and  print  out  the  info  log 
glLinkProgramARB (program) ; 
printOpenGLError ( ) ; 

glGetOb j  ectParameterivARB (program, 

GL_OBJECT_LINK_STATUS_ARB,  Slinked) ; 
printInfoLog (program) ; 

if  ( ! linked) 
return  0; 

return  program; 

} 
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// - 

/ /  Function  :  printOglError 

//  Description: 

//  Returns  1  if  an  OpenGL  error  occurred,  0  otherwise. 

// - 

int  printOglError (char  *file,  int  line) 

{ 

// 

//  Returns  1  if  an  OpenGL  error  occurred,  0  otherwise. 

// 

GLenum  glErr; 

int  retCode  =  0; 

glErr  =  glGetError ( ) ; 
while  (glErr  !=  GL_NO_ERROR) 

{ 

printf ( "glError  in  file  %s  @  line  %d:  %s\n",  file,  line, 
gluErrorString (glErr ) ) ; 
retCode  =  1; 
glErr  =  glGetError () ; 

} 

return  retCode; 


// - 

/ /  Function  :  CreateRenderTexture 

//  Description: 

// - 

RenderTexture*  CreateRenderTexture (const  char  *initstr) 

{ 

printf (" \nCreating  with  init  string:  \"%s\"\n",  initstr) ; 

int  texWidth  =  512,  texHeight  =  512; 

RenderTexture  *rt  =  new  RenderTexture () ; 
rt->Reset (initstr) ; 

if  (! rt->Initialize (texWidth,  texHeight)) 

{ 

fprintf ( stderr ,  "RenderTexture  Initialization  failed! \n"); 

} 

//  for  shadow  mapping  we  still  have  to  bind  it  and  set  the  correct 
//  texture  parameters  using  the  SGI_shadow  or  ARB_shadow  extension 
/ /  setup  the  rendering  context  for  the  RenderTexture 
if  ( rt->BeginCapture ( ) ) 

{ 

pModelFigure->reloadTextures () ; 
pModelGround->reloadTextures () ; 

Reshape (texWidth,  texHeight); 
glClearColor  ( 0 . 5 ,  0.2,  0.2,  1); 

glClear (GL_COLOR_BUFFER_BIT  |  GL_DEPTH_BUFFER_BIT) ; 
rt->EndCapture ( ) ; 

} 


72 


bShowDepthTexture  =  false; 


printOpenGLError ( ) ; 
return  rt; 

} 


// - 

//  Function  : 

//  Description 

DestroyRenderTexture 

// - 

void  DestroyRenderTexture (RenderTexture  *rt 

{ 

delete  rt; 

} 


// - 

//  Function  ; 

//  Description 

Keyboard 

// - 

void  Keyboard (unsigned  char  key,  int  x,  int  y) 
{ 

switch ( key) 

{ 


case  ' a ' : 

eyeX  += 
break; 
case  ' d ' : 

eyeX  -= 
break; 
case  ' q ' : 

exit  ( 0 ) ; 
break; 
default : 

return; 

} 

} 

2; 

2; 

// - 

//  Function  : 

//  Description 

// - 

void  Idle ( ) 

Idle 

{ 

glutPostRedisplay ( )  ; 

} 
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// - 

/ /  Function  :  Reshape 

//  Description  : 

// - 

void  Reshape (int  w,  int  h) 

{ 

if  (h  ==  0)  h  =  1; 

glViewport ( 0 ,  0,  w,  h) ; 

glMatrixMode (GL_PROJECTION) ; 
glLoadldentity ( ) ; 

gluPerspective ( 60 . 0 ,  (GLf loat) w/ (GLf loat) h,  0.1,  5000.0); 

} 

// - 

/ /  Function  :  sumReduce 

//  Description  ; 

// - 

int  sumReduce (  RenderTexture*  rt  ) 

{ 

//  Get  Application  Window  Width  and  Height 
int  ww  =  glutGet (GLUT_WINDOW_WIDTH) ; 
int  wh  =  glutGet (GLUT_WINDOW_HEIGHT) ; 

/ /  Enable  Reduction  Shader 
glUseProgramOb j ectARB (reduceProg) ; 

//  Set  Inputs  to  Shader 
glActiveTexture (GL_TEXTURE0 ) ; 
rt->Bind ( ) ; 

glUniformliARB (glGetUniformLocationARB (reduceProg,  " InputTexture" ) ,  0) ; 
glUniformlfARB (glGetUniformLocationARB (reduceProg,  "Offset") ,  1.0/ww) ; 

//  Set  OpenGL  State 
glClearColor ( 0 . 0 ,  0.0,  0.0,  1); 

glClear (GL_COLOR_BUFFER_BIT  |  GL_DEPTH_BUFFER_BIT) ; 

glMatrixMode (GL_MODELVIEW) ; 

glPushMatrix ( ) ; 

glLoadldentity ( ) ; 

glMatrixMode (GL_PROJECTION) ; 

glPushMatrix ( ) ; 

glLoadldentity ( ) ; 

glOrtho  (0.0,  (float) ww, 0.0,  (float)wh,-1.0,1.0); 
int  Nt  =  ww; 

//  Successively  half  the  problem  size  until  2x2  pixels 
for (int  scale=l;ww/scale>2; scale*=2)  { 

//  Calculate  Vertex  Indices 

int  nv  =  ww/ (2.0*scale);  //  Pix  used  in  viewport 
int  nt  =  ww/scale;  //  Pix  used  in  texture 

/ /  Determine  Texture  Coordinate  Indices 
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float  ta  =  ( (float) nv- (float) nt+1 . 0) /( (float) nv-1 . 0) / (2 . 0* (float) Nt) 
float  tb  =  (1.0  +  (2 . 0* (float) nv-1 . 0) *( (float) nt-2 . 0) / 

( (float) nv-1 .0)  )/ (2.0* (float )Nt); 

//  Reset  OpenGL  State 
glClearColor  ( 0 . ,  0.,  0.,  1); 

glClear (GL_COLOR_BUFFER_BIT  |  GL_DEPTH_BUFFER_BIT) ; 

/ /  Enable  Render  To  Texture 
rt->EnableTextureTarget ( ) ; 

/ /  Draw  Quad  and  Execute  Reduction  Shader 
glEnable (GL_TEXTURE_2D) ; 
glBegin (  GL_QUADS  ) ; 

glTexCoord2f (ta,  ta) ; 
glVertex2f (0,  0); 
glTexCoord2f (ta,  tb) ; 
glVertex2f (0,  nv) ; 
glTexCoord2f (tb,  tb) ; 
glVertex2f (nv,  nv) ; 
glTexCoord2f (tb,  ta) ; 
glVertex2f (nv,  0); 
glEnd ( ) ; 

glDisable (GL_TEXTURE_2D) ; 

//  Copy  the  results  from  the  frame  buffer  to  rt 
glCopyTexSubImage2D (GL  TEXTURE  2D,  0,  0,  0,  0,  0,  nv,  nv) ; 


//  Get  Last  2x2  Pixels 
//  Assumes  GL_PACK_ALIGNMENT  is  4 
for(int  k;=0;  k<16;  k++)  buf[k]=0; 

glReadPixels (0, 0 , 2 , 2 , GL_RGB, GL_UNSIGNED_BYTE, buf ) ; 

//  Repack 

for(k=6;  k<12;  k++)  buf [ k] =buf [ k+2 ] ; 

//  Uncomment  this  to  output  the  results  of  the  reduction 
/*  printf ( "After  reduction:  %d  %d  %d  %d  %d  %d  %d  %d  %d  %d  %d  %d\n", 
buf [0] ,buf [1] ,buf [2] ,buf [3] ,buf [4] ,buf [5] , 
buf [6] ,buf [7] ,buf [8] ,buf [9] , buf [10] , buf [11] ) ; 

*/ 

//  Sum  Visible  Pixels  from  the  Final  Four  pixels 
int  nVisible  =  (buf [0] +buf [3] +buf [6] +buf [9] ) *65536 
+  (buf [l]+buf [4]+buf [7]+buf [10] ) *256 
+  (buf [2]+buf [5]+buf [8]+buf [11] ) ; 

//  Reset  OpenGL  State 
glPopMatrix ( ) ; 
glMatrixMode (GL_MODELVIEW) ; 
glPopMatrix ( ) ; 

/ /  Check  for  OpenGL  Errors 
printOpenGLError ( ) ; 
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//  Disable  Shader 
glUseProgramOb j ectARB ( 0 ) ; 

return  nVisible; 

} 


// - 

//  Function  :  Display 

//  Description  : 

// - 

void  Display  0 

{ 

//  Set  Application  Window  Height  and  Width 
int  ww  =  glutGet (GLUT_WINDOW_WIDTH) ; 
int  wh  =  glutGet (GLUT_WINDOW_HEIGHT) ; 

//  Initialize  Timer  Baseline 
LARGE_INTEGER  start; 

QueryPerf ormanceCounter (Sstart) ; 

//  Enable  Whole  Scene  Pbuffer 

if  (rtAll->IsInitialized ( )  &&  rtAll->BeginCapture ( ) ) 

{ 

//  Set  OpenGL  State 

if  (rtAll->IsDoubleBuffered ( ) )  glDrawBuf fer (GL  BACK); 
glEnable (GL_DEPTH_TEST) ;  ~ 

glClearColor ( 0 . 0 ,  0.0,  0.0,  1); 

glClear (GL_COLOR_BUFFER_BIT  |  GL_DEPTH_BUFFER_BIT) ; 
glMatrixMode (GL_MODELVIEW) ; 
glPushMatrix ( ) ; 
glLoadldentity ( ) ; 

gluLookAt (eyeX,  0,  -100,  0,  0,  0,  0,  1,  0); 

//  Render  figure  +  ground 
pModelGround->draw ( ) ; 
pModelFigure->draw ( ) ; 

//  Reset  OpenGL  State  and  Disable  Pbuffer 
glPopMatrix ( ) ; 
printOpenGLError ( ) ; 
rtAll->EndCapture ( ) ; 

} 

/ /  Calculate  Whole  Scene  Render  Time 
LARGE_INTEGER  allTime; 

QueryPerf ormanceCounter ( SallTime) ; 
allTime . QuadPart  -=  start . QuadPart; 


//  Reset  Timer  Baseline 
QueryPerf ormanceCounter ( Sstart)  ; 
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//  Enable  Figure-Only  Pbuffer 
if  (rtFig->IsInitialized ( )  &&  rtFig->BeginCapture ( ) ) 

{ 

//  Set  OpenGL  State 

if  (rtFig->IsDoubleBuf fered ( ) )  glDrawBuf fer (GL  BACK); 
glMatrixMode (GL_MODELVIEW) ;  ~ 

glPushMatrix ( ) ; 
glLoadldentity ( ) ; 

gluLookAt (eyeX,  0,  -100,  0,  0,  0,  0,  1,  0); 
glClearColor ( 0 . 0 ,  0.0,  0.5,  1); 

glClear (GL_COLOR_BUFFER_BIT  |  GL_DEPTH_BUFFER_BIT) ; 

//  Render  figure  only 
pModelFigure->draw ( ) ; 

//  Reset  OpenGL  State  and  Disable  Pbuffer 
glPopMatrix ( ) ; 
printOpenGLError ( ) ; 
rtFig->EndCapture ( ) ; 

} 

/ /  Calculate  Figure-Only  Render  Time 
LARGE_INTEGER  figTime; 

QueryPerf ormanceCounter ( &f IgTime) ; 
figTime . QuadPart  -=  start . QuadPart; 


//  Reset  Timer  Baseline 
QueryPerf ormanceCounter ( Sstart) ; 

//  Enable  Visualization  Pbuffer 

if  (rtVis->IsInitialized ( )  &&  rtVis->BeginCapture ( ) ) 

{ 

//  Activate  Visualization  Shader 
glUseProgramOb j ectARB (visTexProg) ; 

//  Set  Inputs  to  Shader 
glActiveTexture (GL_TEXTURE2 ) ; 
rtAll->Bind() ; 

glActiveTexture (GL_TEXTURE3 ) ; 
rtFig->Bind ( ) ; 

glUniformliARB (glGetUniformLocationARB (visTexProg,  "AllTexture" ) ,  2 ) ; 
glUniformliARB (glGetUniformLocationARB (visTexProg,  "FigureTexture" ) ,  3) 
glUnif orm3f ARB (glGetUniformLocationARB (visTexProg,  "backgroundColor" ) , 

0.4,  0.0,  0.0); 

//  Set  OpenGL  State 

if  (rtVis->IsDoubleBuffered ( ) )  glDrawBuf fer (GL  BACK); 

glDisable (GL_DEPTH_TEST) ;  ~ 

glMatrixMode (GL_MODELVIEW) ; 

glPushMatrix ( ) ; 

glLoadldentity ( ) ; 

glMatrixMode (GL_PROJECTION) ; 

glPushMatrix ( ) ; 

glLoadldentity ( ) ; 

glOrtho (0.0, (float) ww, 0.0, (float)wh,-1.0,1.0); 
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/ /  Create  Quad  Spanning  Viewport  and  Run  Shader 
glEnable (GL_TEXTURE_2D) ; 
glBegin (  GL_QUADS  ) ; 
glTexCoordCf (0,  0); 
glVertex2f (0,  0); 
glTexCoord2f (0,  1); 
glVertex2f (0,  wh) ; 
glTexCoord2f ( 1 ,  1 ) ; 
glVertex2f (ww,  wh) ; 
glTexCoord2f ( 1 ,  0); 
glVertex2f (ww,  0); 
glEnd ( ) ; 

//  Reset  OpenGL  State 
glDisable (GL_TEXTURE_2D) ; 
glPopMatrix ( ) ; 
glMatrixMode (GL_MODELVIEW) ; 
glPopMatrix ( ) ; 
glEnable (GL_DEPTH_TEST) ; 
printOpenGLError ( ) ; 

//  Disable  Pbuffer  and  Shader 
rtVis->EndCapture ( ) ; 
glUseProgramOb j ectARB ( 0 ) ; 


//  Calculate  Visualization  Render  Time 
LARGE_INTEGER  visTime; 

QueryPerf ormanceCounter ( SvisTime) ; 
visTime . QuadPart  -=  start . QuadPart; 


//  Reset  Timer  Baseline 
QueryPerf ormanceCounter (Sstart) ; 

//  Enable  Comparison  Pbuffer 

if  (rtRed->IsInitialized ( )  &&  rtRed->BeginCapture ( ) ) 

{ 

//  Enable  Comparison  Shader 
glUseProgramObj ectARB (visBoolProg) ; 

//  Set  up  Shader  Inputs 
glActiveTexture (GL_TEXTURE2 ) ; 
rtAll->Bind ( ) ; 

glActiveTexture (GL_TEXTURE3 ) ; 
rtFig->Bind ( ) ; 

glUniformliARB (glGetUnif ormLocationARB (visBoolProg, 
glUniformliARB (glGetUnif ormLocationARB (visBoolProg, 
glUnif orm3f ARB (glGetUnif ormLocationARB (visBoolProg, 

0.0,  0.0,  0. 


"AllTexture") ,  2 ) ; 
"FigureTexture" ) ,  3) 
"backgroundColor" ) , 

)  ; 
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//  Set  OpenGL  State 

if  (rtRed->IsDoubleBuf fered ( )  )  glDrawBuf fer (GL  BACK) 

glMatrixMode (GL_MODELVIEW) ;  ~ 

glPushMatrix ( ) ; 

glLoadldentity ( ) ; 

glMatrixMode (GL_PROJECTION) ; 

glPushMatrix ( ) ; 

glLoadldentity ( ) ; 

glOrtho (0.0, (float) ww, 0.0, (float)wh,-1.0,1.0); 

//  Create  Quad  and  Run  Shader 
glEnable (GL_TEXTURE_2D) ; 
glBegin (  GL_QUADS  ) ; 
glTexCoord2f (0,  0); 
glVertex2f (0,  0); 
glTexCoord2f (0,  1); 
glVertex2f (0,  wh) ; 
glTexCoord2f ( 1 ,  1 ) ; 
glVertex2f (ww,  wh) ; 
glTexCoord2f ( 1 ,  0); 
glVertex2f (ww,  0); 
glEnd ( ) ; 

glDisable (GL_TEXTURE_2D) ; 

//  Reset  OpenGL  State 
glPopMatrix ( ) ; 
glMatrixMode (GL_MODELVIEW) ; 
glPopMatrix ( ) ; 
printOpenGLError ( ) ; 

//  Disable  Comparison  Pbuffer  and  Shader 
rtRed->EndCapture () ; 
glUseProgramOb j ectARB ( 0 ) ; 


//  Calculate  Comparison  Time 
LARGE_INTEGER  redTime; 

QueryPerf ormanceCounter ( SredTime) ; 
redTime . QuadPart  -=  start . QuadPart; 


//  Do  Timed  Reduction  on  rtRed 
QueryPerf ormanceCounter ( Sstart) ; 

int  nVisible  =  sumReduce (  rtRed  ) ; 

printf ( "Number  of  visible  pixels  l:\t%d\n",  nVisible); 

LARGE_INTEGER  reductionlTime; 

QueryPerf ormanceCounter ( SreductionlTime) ; 
reductionlTime . QuadPart  -=  start . QuadPart; 


//  Reset  Timer  Baseline 
QueryPerf ormanceCounter ( Sstart) ; 
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//  Enable  False  Color  Whole  Scene  Pbuffer 
if  (rtA112->IsInitialized ( )  &&  rtA112->BeginCapture ( ) ) 

{ 

//  Set  OpenGL  State 

if  (rtA112->IsDoubleBuf fered ( ) )  glDrawBuf fer (GL  BACK); 
glEnable (GL_DEPTH_TEST) ;  ~ 

glClearColor ( 0 . 0 ,  0.0,  0.0,  1); 

glClear (GL_COLOR_BUFFER_BIT  |  GL_DEPTH_BUFFER_BIT) ; 
glMatrixMode (GL_MODELVIEW) ; 
glPushMatrix ( ) ; 
glLoadldentity ( ) ; 

gluLookAt (eyeX,  0,  -100,  0,  0,  0,  0,  1,  0); 

//  Render  figure  +  ground 
pModelGround->drawFlat (0.5, 0.5, 0.5); 
pModelFigure->drawFlat (1.0, 1.0, 1.0) ; 

//  Reset  OpenGL  State  and  Disable  False  Color  Pbuffer 
glPopMatrix ( ) ; 
printOpenGLError ( ) ; 
rtA112->EndCapture ( ) ; 


//  Calculate  False  Color  Render  Time 
LARGE_INTEGER  all2Time; 

QueryPerf ormanceCounter ( &all2Time) ; 
all2Time . QuadPart  -=  start . QuadPart; 


//Reset  Timer  Baseline 
QueryPerf ormanceCounter ( Sstart)  ; 

//  Enable  False  Color  Comparison  Pbuffer 
if  (rtRed2->IsInitialized ( )  &&  rtRed2->BeginCapture ( ) ) 

{ 

//  Enable  Comparison  Shader 
glUseProgramOb j ectARB (visBool2Prog) ; 

//  Set  Shader  Inputs 
glActiveTexture (GL_TEXTURE0 ) ; 
rtA112->Bind ( ) ; 

glUniformliARB (glGetUniformLocationARB (visBool2Prog,  "AllTexture" ) ,  0) 
glUnif ormSf ARB (glGetUniformLocationARB (visBool2Prog,  "f igureColor" ) , 
1.0,  1.0,  1.0); 

//  Set  OpenGL  State 

if  (rtRed2->IsDoubleBuf fered 0 )  glDrawBuf fer (GL  BACK); 

glClearColor ( 0 . 0 ,  0.0,  0.0,  1); 

glClear (GL_COLOR_BUFFER_BIT  ); 

glViewport ( 0 ,  0,  ww,  wh) ; 

glMatrixMode (GL_MODELVIEW) ; 

glPushMatrix ( ) ; 

glLoadldentity ( ) ; 

glMatrixMode (GL_PROJECTION) ; 

glPushMatrix ( ) ; 

glLoadldentity ( ) ; 

glOrtho (0.0, (float) ww, 0.0, (float)wh,-1.0,1.0); 
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/ /  Draw  Quad  and  Run  Shader 
glEnable (GL_TEXTURE_2D) ; 
glBegin (  GL_QUADS  ) ; 
glTexCoord2f (0,  0); 
glVertex2f (0,  0); 
glTexCoord2f (0,  1); 
glVertex2f (0,  wh) ; 
glTexCoord2f ( 1 ,  1 ) ; 
glVertex2f (ww,  wh) ; 
glTexCoord2f ( 1 ,  0); 
glVertex2f (ww,  0); 
glEnd ( ) ; 

glDisable (GL_TEXTURE_2D) ; 

//  Reset  OpenGL  State 
glPopMatrix ( ) ; 
glMatrixMode (GL_MODELVIEW) ; 
glPopMatrix ( ) ; 
printOpenGLError ( ) ; 

//  Disable  Comparison  Pbuffer  and  Shader 
rtRed2->EndCapture () ; 
glUseProgramOb j ectARB ( 0 ) ; 


/ /  Calculate  Red2  RenderTime 
LARGE_INTEGER  red2Time; 

QueryPerf ormanceCounter ( &red2Time) ; 
red2Time . QuadPart  -=  start . QuadPart; 


//  Do  Timed  Redcution  on  rtRed2 
QueryPerf ormanceCounter (Sstart) ; 

int  nVisible2  =  sumReduce (  rtRed2  ) ; 

printf ( "Number  of  visible  pixels  2:\t%d\n",  nVisible2) 

LARGE_INTEGER  reduction2Time; 

QueryPerf ormanceCounter ( &reduction2Time) ; 
reduction2Time . QuadPart  -=  start . QuadPart; 


//  Begin  render  visible  scene 

/ /  Set  OpenGL  State  to  Render  Main  Window 

glClear (GL_COLOR_BUFFER_BIT  |  GL_DEPTH_BUFFER_BIT) ; 

glMatrixMode (GL_MODELVIEW) ; 

glPushMatrix ( ) ; 

glLoadldentity ( ) ; 

gluLookAt (eyeX,  0,  -100,  0,  0,  0,  0,  1,  0); 

/ /  Draw  Figure  +  Ground 
pModelGround->draw ( ) ; 
pModelFigure->draw ( ) ; 
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//  Set  OpenGL  State  to  Render  Smaller  Views 

glMatrixMode (GL_MODELVIEW) ; 

glPushMatrix ( ) ; 

glLoadldentity ( ) ; 

glMatrixMode (GL_PROJECTION) ; 

glPushMatrix ( ) ; 

glLoadldentity ( ) ; 

glOrtho (0.0, (float) ww, 0.0, (float)wh,-1.0,1.0); 

float  of  =  0.01;  //  offset  fraction 

float  sf  =  0.1;  //  size  fraction 

float  space=ww/5; 

/ /  Bind  Textures  to  Rectangles 
for(int  i=0;i<4;i++)  { 

if(i==0)  {  //  Bind  Whole  Scene  Pbuffer 

rtAll->Bind ( ) ; 

if  (rtAll->IsDoubleBuf fered ( ) )  rtAll->BindBuf fer (WGL  BACK  LEFT  ARB) 
rtAll->EnableTextureTarget ( ) ; 

} 

if(i==l)  {  //  Bind  Figure-Only  Pbuffer 
rtFig->Bind ( ) ; 

if  (rtFig->IsDoubleBuf fered 0 )  rtFig->BindBuf fer (WGL  BACK  LEFT  ARB) 
rtFig->EnableTextureTarget () ; 

} 

if(i==2)  {  //  Bind  Occluded  Figure-Only  Pbuffer 

rtVis->Bind ( ) ; 

if  (rtVis->IsDoubleBuf fered 0 )  rtVis->BindBuf fer (WGL  BACK  LEFT  ARB) 
rtVis->EnableTextureTarget ( ) ; 

} 

if(i==3)  {//  Bind  False  Color  Whole  Scene  Pbuffer 

rtA112->Bind ( ) ; 

rtA112->EnableTextureTarget ( ) ; 

} 

//  Draw  Rectangle  with  Bound  Texture  (Pbuffer) 
glEnable (GL_TEXTURE_2D) ; 
glBegin (  GL_QUADS  ) ; 
glTexCoord2f (0,  0); 
glVertex2f (space*i+of *ww,  of*wh) ; 
glTexCoord2f (0,  1); 

glVertex2f (space*i+of *ww,  (of+sf) *wh) ; 
glTexCoord2f ( 1 ,  1); 

glVertex2f (space*!  +  (of+sf) *ww,  (of+sf) *wh) ; 
glTexCoord2f ( 1 ,  0); 

glVertex2f (space*!  +  (of+sf ) *ww,  of*wh) ; 
glEnd ( ) ; 

glDisable (GL_TEXTURE_2D) ; 
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//  Disable  Pbuffer 
if(i==0)  { 

rtAll->DisableTextureTarget  ( ) ; 

} 

if(i==l)  { 

rtFig->DisableTextureTarget () ; 

} 

if(i==2)  { 

rtVis->DisableTextureTarget ( ) ; 

} 

if(i==3)  { 

rtA112->DisableTextureTarget ( ) ; 

} 


//  Reset  OpenGL  State 
glPopMatrix ( ) ; 
glMatrixMode (GL_MODELVIEW) ; 
glPopMatrix ( ) ; 
glPopMatrix ( ) ; 


//  Reset  Timer  Baseline 
QueryPerf ormanceCounter (Sstart) ; 


//  UNCOMMENT  to  Render  False  Color  to  screen  and 
//  use  ReadPixels  to  count  pixels 
/*  glClearColor(0.0,0.0,0.0,1.0) ; 

glClear (GL_COLOR_BUFFER_BIT  |  GL_DEPTH_BUFFER_BIT) 

glMatrixMode (GL_MODELVIEW) ; 

glPushMatrix ( ) ; 

glLoadldentity ( ) ; 

glMatrixMode (GL_PROJECTION) ; 

glPushMatrix ( ) ; 

glLoadldentity ( ) ; 

glOrtho (0.0, (float) ww, 0.0, (float)wh,-1.0,1.0); 

//  Bind  False  Color  Whole  Scene 
rtA112->Bind ( ) ; 

rtA112->EnableTextureTarget ( ) ; 

/ /  Draw  Main  Window 
glEnable (GL_TEXTURE_2D) ; 
glBegin (  GL_QUADS  ) ; 
glTexCoord2f (0,  0); 
glVertex2f (0,  0); 
glTexCoord2f (0,  1); 
glVertex2f (0,  wh) ; 
glTexCoord2f ( 1 ,  1 ) ; 
glVertex2f (ww,  wh) ; 
glTexCoord2f ( 1 ,  0); 
glVertex2f (ww,  0); 
glEnd ( ) ; 

glDisable (GL_TEXTURE_2D) ; 
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//  Reset  OpenGL  State 
glPopMatrix ( ) ; 
glMatrixMode (GL_MODELVIEW) ; 
glPopMatrix ( ) ; 

int  nv  =  ww; 
int  sum_gray  =  0; 
int  sum  white  =  0; 
int  sum_black;  =  0; 

//  Reset  buffer 

for(int  j=0; j<nv*nv*3; j++)  buf[j]=0; 

//  Fill  buffer  with  pixels  from  framebuffer 
glReadPixels (0, 0, nv, nv, GL_RGB, GL_UNSIGNED_BYTE, buf ) ; 

//  Count  White,  Grey,  and  Black  Pixels 
for (int  i=0; i<nv; i++)  { 

for ( j=0; j<nv; j++)  { 

int  blue  =  buf [i*nv*3+j *3+2] ; 
int  green  =  buf [i*nv*3+j *3+1] ; 
int  red  =  buf [i*nv*3+j *3+0] ; 
if (red==255  &&  green==255  &&  blue==255) 

++sum_white; 

else  if(red==128  &&  green==128  &&  blue==128) 

++sum_gray; 

else  if(red==0  &&  green==0  &&  blue==0) 

++sum_black; 

else  {  //  Some  Other  Color  (error) 

printf("(%d  %d  %d)  ",  red, green, blue) ; 

) 

} 

} 

//printf  ("\nNumber  of  white  pixels  in  rtA112:  %d  Gray  pixels:  %d  ''  + 
//  "Black:  %d  Total:  %d  \n",  sum_white,  sum_gray, 

/  /  sum_black,  sum_white+sum_gray+sum_black) ; 

//printf ( "Number  of  visible  pixels  3 : \t%d\n" , sum_white) ; 

//  Calculate  Count  Time 
LARGE_INTEGER  countTime; 

QueryPerf ormanceCounter (ScountTime) ; 
countTime . QuadPart  -=  start . QuadPart; 


//  Swap  Frame  Buffers  to  display 
printOpenGLError ( ) ; 
glutSwapBuf fers () ; 

//  Get  This  Frame  Time 
LARGE_INTEGER  frameTime; 

QueryPerf ormanceCounter ( SframeTime) ; 

//  Find  Unit  Frequency  (  1/freq  units  per  second  ) 
LARGE_INTEGER  freq; 

QueryPerf ormanceFrequency ( &f req) ; 
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/ /  Determine  Overhead  Involved  in  Querying  Time 
LARGE_INTEGER  Ctrl; 

LARGE_INTEGER  ctr2; 

LARGE_INTEGER  overhead; 

QueryPerf ormanceCounter ( &ctrl ) ; 

QueryPerf ormanceCounter ( &ctr2 ) ; 

overhead . QuadPart  =  ctr2 . QuadPart  -  Ctrl . QuadPart; 

//  Accumulate  Times  and  Frames  (ticks) 
ticks++; 

accumReductionlTime  +=  (double) (reductionlTime . QuadPart  -  overhead. QuadPart) 

/  (double) (freq. QuadPart) ; 

accumReduction2Time  +=  (double) (reduction2Time . QuadPart  -  overhead. QuadPart) 

/  (double) (freq. QuadPart) ; 

accumCountTime  +=  (double) (countTime . QuadPart  -  overhead. QuadPart) 

/  (double) (freq. QuadPart) ; 

accumFrameTime  +=  (double) ( frameTime . QuadPart  -  lastFrame . QuadPart 

-  overhead. QuadPart)  /  (double) (freq. QuadPart) ; 

lastFrame  =  frameTime; 

/ /  Calculate  Average  Times 

double  avgRedTime  =  accumReductionlTime  /  (double) ticks; 
double  avgRed2Time  =  accumReduction2Time  /  (double) ticks; 
double  avgCountTime  =  accumCountTime  /  (double) ticks; 
double  avgFrameTime  =  accumFrameTime  /  (double) ticks; 


// - 

/ /  Function  ;  main 

//  Description  : 

// - 

void  main ( ) 

{ 

//  Create  Container  for  Shader  Source  (to  load  shaders) 

GLcharARB  * Fragment Shader Source; 

//  Initialize  Window 

glutInitDisplayMode (GLUT_RGB  |  GLUT_DEPTH  |  GLUT_DOUBLE) ; 
glutInitWindowPosition (50,  50); 
glutInitWindowSize (512,  512); 
glutCreateWindow ( "TestRenderTexture" ) ; 

/ /  Check  for  OpenCL  Errors 
int  err  =  glewInitO; 
if  (GLEW_OK  !=  err) 

{ 

//  problem:  glewinit  failed,  something  is  seriously  wrong 
fprintf ( stderr ,  "CLEW  Error:  %s\n",  glewGetErrorString (err) ) ; 
exit  (-1) ; 

} 
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//  Ensure  Required  OpenGL  Extensions  are  Supported 

if  (  ! isExtensionSupported (  "GL  ARB  shader  objects"  )  | | 

! isExtensionSupported (  "GL  ARB  fragment  shader"  )  | | 

! isExtensionSupported (  "GL  ARB  vertex  shader"  )  | | 

! isExtensionSupported (  "GL  ARB  shading  language  100"  )  ) 

{ 

printf ( "OpenGL  Shading  Language  extensions  not  available\n"  ) ; 
exit (-1) ; 

} 

//  Register  Functions  to  be  executed  in  glutMainLoop 

glutDisplayFunc (Display) ; 

glutIdleFunc (Idle) ; 

glutReshapeFunc (Reshape) ; 

glutKeyboardFunc (Keyboard) ; 

//  Ensure  Window  is  512x512 
Reshape (512,  512); 

//  Initialize  Container  for  Retrieved  Pixels 
buf  =  new  unsigned  char [512*512*3] ; 

//  Setup  Main  App ' s  OpenGL  State 
glMatrixMode (GL_MODELVIEW) ; 
glLoadldentity ( ) ; 

gluLookAt (eyeX,  0,  -100,  0,  0,  0,  0,  1,  0); 
glDisable (GL_LIGHTING) ; 
glEnable (GL_COLOR_MATERIAL) ; 
glEnable (GL_DEPTH_TEST) ; 
glClearColor ( 0 . 4 ,  0.6,  0.8,  1); 

//  Output  Projection  Matrix 
float  m [ 1 6 ] ; 

glGetFloatv (GL_PROJECTION_MATRIX, m) ; 
for(int  i=0; i<16; i++)  printf ("%f  ",m[i]); 
printf ("\n") ; 

/ /  Load  Model  of  Background  and  Obstacle 
pModelGround  =  new  MilkshapeModel ( ) ; 

if  (  pModelGround->loadModelData (  "data/ground. ms3d"  )  ==  false  ) 

{ 

fprintf  (  stderr,  "Couldn't  load  the  model  dataWmodel  .ms3d")  ; 
exit (  -1  ) ; 

} 

//  Load  Model  of  Figure  (target) 
pModelFigure  =  new  MilkshapeModel () ; 

if  (  pModelFigure->loadModelData (  "data/figure .ms3d"  )  ==  false  ) 

{ 

fprintf  (  stderr,  "Couldn't  load  the  model  dataWmodel  .ms3d")  ; 
exit (  -1  ) ; 

} 
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//  Create  and  Initialize  RenderTextures 

rtAll  =  CreateRenderTexture ( "rgb  tex2D  depthTex2D" ) ; 

rtFig  =  CreateRenderTexture ( "rgb  tex2D  depthTex2D" ) ; 

rtVis  =  CreateRenderTexture ( "rgb  tex2D"); 

rtRed  =  CreateRenderTexture ( "rgb  tex2D"); 

rtA112  =  CreateRenderTexture ( "rgb  tex2D  depthTex2D" ) ; 

rtRed2  =  CreateRenderTexture ( "rgb  tex2D"); 

/ /  Load  OpenGL  Shading  Language  Shaders 

readShaderSource ( "colorbasedvis . frag" ,  SFragmentShaderSource) ; 
visTexProg  =  installShader (FragmentShaderSource) ; 

readShaderSource ( "boolvis . frag" ,  SFragmentShaderSource) ; 
visBoolProg  =  installShader (FragmentShaderSource) ; 

readShaderSource ( "sumreduce . frag" ,  SFragmentShaderSource) ; 
reduceProg  =  installShader (FragmentShaderSource) ; 

readShaderSource ( "boolvis2 . frag" ,  SFragmentShaderSource) ; 
visBool2Prog  =  installShader (FragmentShaderSource) ; 

printf ( "Press  a  or  d  to  sidestep  left  or  right.  Press  q  to  quit.\n") 

//  Register  first  Time  for  Timing  Frames 
QueryPerf ormanceCounter ( SlastFrame) ; 

//  And  awaaaay  we  go... 
glutMainLoop ( ) ; 
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GLOSSARY 


Callback 

A  method  which  is  registered  with  a  scene  graph  node  and  is  called  by 
that  node  when  it  is  traversed  by  the  corresponding  traversal.  For 
example,  a  cull  callback  can  be  registered  with  a  node  to  execute  when 
that  node  is  traversed  by  a  cull  traversal.  Some  common  callbacks  are 
cull,  draw,  and  update.  Some  scene  graphs  also  implement  pre-  and 
post-  callbacks  for  execution  before  and  after  traversing  the  node. 

Classification 

Discrimination  between  general  classes  of  vehicles  (tracked,  wheeled, 
etc.).  [NVESD1] 

Clip  Space 

“A  coordinate  space  that  is  suitable  for  clipping.”  [Rost04]  In  the  rendering 
pipeline,  objects  are  taken  from  the  viewing  volume  to  clip  space  by  an 
application  of  the  Projection  Transformation. 

Culling 

The  removal  of  an  object  or  objects  from  the  list  of  objects  to  be  rendered 
because  they  lie  outside  the  view  frustum. 

Depth  Buffer 

Also  referred  to  as  the  Z  Buffer.  A  component  of  the  frame  buffer  which, 
just  as  the  color  buffer  holds  the  color  information  of  each  pixel,  holds  the 
depth  of  each  pixel.  The  depth  buffer,  through  depth  testing,  is  used  to 
determine  if  the  current  pixel  is  in  front  or  behind  any  previous  pixel  written 
at  the  same  window  position. 

Detection 

Determination  that  a  target  is  present  within  a  field-of-view.  [NVESD1] 

DirectX 

Microsoft  DirectX  is  a  suite  of  multimedia  API’s  built  into  the  Microsoft 
Windows  operating  system. 

Eye  Space 

A  coordinate  space  relative  to  the  observer  or  camera’s  viewpoint.  An 
object  is  transformed  from  object  space  to  eye  space  by  application  the 
modelview  matrix. 
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Fragment 

A  discrete  unit  of  area  on  the  interior  of  a  primitive.  Its  color  and  depth 
values  are  interpolated  from  the  values  of  the  vertices  of  that  primitive.  It 
is  put  through  several  tests  by  the  fragment  processor  and  if  it  survives, 
the  fragment  is  passed  to  the  rasterizer  to  be  mapped  to  pixels. 

Fragment  Program 

On  a  programmable  GPU,  a  program  that  is  executed  by  the  fragment 
processor,  replacing  the  OpenGL  “fixed  functionality.” 

Frame  Buffer 

A  section  of  video  memory  dedicated  for  use  to  store  rendering 
information  in  the  form  of  arrays  of  pixels.  The  frame  buffer  comprises 
several  subbuffers:  color,  depth,  stencil,  accumulation,  and  pixel  buffers. 

Frame  Buffer  Memory 

The  portion  of  video  memory  dedicated  for  frame  buffer  use. 

Graphics  Library  Utiiity  Toolkit  (GLUT) 

A  platform-independent  toolkit  for  creating  windowed  applications. 
Documentation  can  be  found  at 

http://www.opengl.org/documentation/specs/glut/spec3/spec3.html 

Graphics  Library  Extension  Wrangler  (GLEW) 

An  API  to  initialize  and  use  OpenGL  extensions.  GLEW  also  provides  a 
means  to  determine  if  particular  extensions  are  available  at  run  time. 
GLEW  was  originally  written  by  Milan  Ikits  and  Marcelo  Magallon. 
[http://glew.sourceforge.net/] 

Identification 

Discrimination  between  specific  targets  (T72,  M1,  Chieftain,  etc.). 
[NVESD1] 

Line  of  Sight 

A  geometric  determination  of  whether  the  view  of  the  target  from  an 
observer  is  unobstructed. 

Load  Balancing 

The  effort  to  equalize  the  loads  of  the  CPU  and  GPU  to  realize  optimal 
performance.  This  is  a  dynamic  effort  as  at  any  moment,  an  application 
can  be  either  CPU-  or  GPU-bound.  Load  balancing  attempts  to  detect 
and  relieve  the  computational  load  of  the  bound  processor. 

Object  Space 

A  coordinate  space  relative  to  an  object.  The  origin  of  that  object  space  is 
coincident  with  the  object’s  origin. 
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Open  Graphics  Library  (OpenGL). 

A  C++  API  for  rendering  objects  to  a  display  device.  It  is  widely  supported 
by  hardware  manufacturers.  OpenGL  is  a  trademark  of  Silicon  Graphics, 
Inc. 


Pixel 

The  color  value  at  a  specific  point  in  a  scene.  Technically,  as  a  point,  a 
pixel  has  no  area.  So,  when  talking  about  processing  pixels  and  pixel 
shaders,  we  are  actually  referring  to  fragments. 

Pixel  Buffer  (PBuffer) 

A  segment  of  graphics  memory  analogous  to  a  frame  buffer.  Unlike  the 
frame  buffer,  the  pixel  buffer  is  not  designed  to  be  displayed  directly  to  the 
user.  It  is  often  used  for  off-screen  rendering  and  rendering  to  texture, 
where  the  pbuffer  can  be  bound  in  a  fashion  similar  to  a  texture. 

Primitive 

A  basic  unit  of  graphics  geometry.  All  geometric  models  are  composed  of 
at  least  one  primitive.  There  are  few  primitives  in  OpenGL:  triangle,  quad, 
and  point.  There  are  also  optimized  groups  of  these  primitives:  triangle 
strip  and  quad  strip.  One  basic  rule  in  OpenGL  is  that  the  vertices  of  a 
primitive  must  be  coplanar.  Otherwise,  results  are  undefined. 

Recognition 

Discrimination  between  categories  within  a  class  of  similar  objects  (tank, 
APC,  self  propelled  howitzer,  etc.).  [NVESD1] 

Render  To  Texture 

A  process  in  which  a  scene  is  rendered  to  a  render  target  and  that  render 
target  is  accessible  as  a  texture.  Currently  render  to  texture  is  only 
supported  in  Microsoft  Window  by  OpenGL.  Other  platforms  must  use  a 
“copy  to  texture”  scheme  where  the  scene  is  rendered  to  a  render  target 
and  then  the  color  information  is  copied  from  that  render  target  to  a 
texture. 

Resolvable  Cycles 

The  number  of  finite  units  of  resolution  that  span  the  critical  dimension  of  a 
target  in  the  electro-optic  sensor’s  field  of  view.  [Johnson58] 

Scene  Graph 

A  tree  composed  of  nodes  representing  objects  and  operations  on  those 
objects  in  a  three-dimensional,  graphical  world.  Operations  on  a  scene 
graph  are  usually  performed  by  traversals,  where  each  node  in  the  tree  is 
acted  upon  only  once  per  traversal. 
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Shader  Object 

A  component  of  a  Shader  Program,  whether  a  vertex  or  fragment  shader 
object.  The  shader  objects  are  compiled  and  linked  to  form  Shader 
Programs.  [Rost04] 

Spot  Detection 

“The  target  spot  detection  (also  referred  to  as  "star"  detection) 
methodology  used  in  ACQUIRE  is  designed  for  cases  in  which  the  target 
is  viewed  against  a  uniform  background.”  [NVESD1] 

Target  Discrimination 

“The  target  discrimination  methodology  is  useful  for  target  detection 
situations  in  which  a  separation  of  the  target  characteristics  from  the 
background  is  required  (e.g.,  when  a  target  is  embedded  in  a  non-uniform 
or  cluttered  background).  The  target  discrimination  methodology  can  be 
used  for  the  prediction  of  greater  levels  of  target  discrimination  beyond 
detection  such  as  classification,  recognition,  and  identification.”  [NVESD1] 

Texture 

An  image  which  can  be  applied  to  the  surface  of  an  object.  Textures  can 
be  images  read  from  files  or  can  be  generated  procedurally.  Because 
textures  can  be  created  procedurally  and  can  be  used  as  inputs  to  shader 
programs,  they  are  used  to  provide  array  of  inputs  to  shader  objects. 

Texture  Memory 

Video  card  memory  dedicated  to  the  storage  of  texture  information.  This 
memory  is  finite,  so  if  more  textures  are  required  in  a  scene  than  can  be 
stored  in  texture  memory,  the  textures  are  paged  in  and  out  of  texture 
memory. 

Traversal 

In  a  scene  graph,  a  visiting  task  in  which  each  node  in  the  scene  graph  is 
acted  upon  only  once.  Cull,  Draw,  and  Update  are  three  examples  of 
common  scene  graph  traversals. 


Vertex 

Similar  to  the  geometric  definition,  a  point  that  describes  a  corner  of  a 
primitive.  Triangles  have  three  vertices,  for  example. 

Vertex  Program 

On  a  programmable  GPU,  a  program  that  is  executed  by  the  vertex 
processor,  replacing  the  OpenGL  “fixed  functionality.” 
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